- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies whether human verification of AI-generated outputs can sustain human skill over time. The authors model a worker who chooses between three options: doing task manually, using AI and checking its outputs or fully delegating the task to AI. Each choice affects future skill through learning or forgetting. The authors show that verification does not always prevent skill loss and workers may gradually lose skill even while checking and correcting AI outputs. It may be likely that skill may collapse, stabilize through verification or be maintained by switching between manual work and verification depending on the conditions. The authors also show that better AI capabilities can lead to greater skill loss by reducing manual practice and opportunities to correct AI mistakes and that random events can push workers from stable verification into full delegation. The paper provides theoretical proofs, numerical examples, sensitivity analyses, and stochastic simulations. It also includes an illustrative mapping from educational data to model parameters, but does not empirically validate the long-term predictions.
Critical strengths
-
This is an important problem about long-term human-AI interaction and its impact. The authors focus on how repeated AI use changes human skill over time instead of focusing on the immediate benefits of AI assistance. This is particularly relevant as human verification is often proposed as a sanity check for reliable AI use. The paper argues that keeping humans in the loop for checking AI outputs does not necessarily retain their skills.
-
The theoretical analysis identifies conditions in which verification can sustain skill, need for manual practice or collapse of a skill. The interesting observation from the work is that even when verification could sustain skill if continued, workers may switch to full delegation before reaching that stable level. The paper also shows that changing workflow incentives alone can lead to different long-term outcomes without changing the learning dynamics.
Critical weaknesses
-
The model uses a single skill variable for both doing the task manually and verifying AI outputs. I think this is a major limitation since the main question of the paper is whether verification can sustain skill. These two skills can develop differently. For example, someone who reviews a lot of AI-generated code may become slower at writing code but better at finding bugs. The current model cannot capture this. Also, since verification and manual work share the same skill, any forgetting from not doing manual work also reduces verification ability.
-
The model doesn't factor in that working with AI is a skill as well which people can get better at over time. Verification can also be done through automated ways such as tests, external tools and not just manual debugging which the current model fails to account.
-
I think the results depend a lot on the learning and forgetting assumptions. Manual work always gives at least as much learning as verification, learning slows down at higher skill levels and forgetting can be larger than what is gained from verification. Under these assumptions, it is expected that even highly skilled workers lose skill while verifying. Section 4 shows that collapse can be avoided with stronger verification learning or weaker forgetting but it is not clear which parameter values are realistic.
-
The result that better AI cannot improve human skill also seems to follow from the model design. So, in this model, the worker learns from verification by a fixed amount that does not depend on better AI and correct AI outputs doesn't necessarily improve learning. Appendix E.2 shows that learning can improve with correct AI outputs under a fixed AI quality. However, it is unclear if the findings hold once when both additional learning and workflow switching are considered.
-
The workers always pick the option with the highest immediate utility and do not think about their future skill. For instance, verification could keep skill at a stable level, but the worker switches to full delegation before getting there and the skill collapses. Real workers may keep verifying to stay competent or reduce their AI use slowly instead of switching all at once. So, I think the collapse mostly comes from this assumption. The proofs may be correct under the assumptions, but I am not convinced that the conclusions apply to real AI-assisted work.
Questions for authors
-
How would the conclusions change if doing the task manually and verifying AI outputs are treated as different skills? Could verification remain effective or improve even as doing task manually skill declines?
-
What supports the assumed balance between verification learning, manual practice, and forgetting? Section 4 shows that changing these rates can prevent collapse so how should we understand which parameter settings are realistic?
-
Under the additional learning mechanism in Appendix E.2, could stronger AI improve long-term human skill when workflow switching is considered? Would this change the conclusion that better AI cannot increase retained skill?
-
How sensitive are the collapse results to workers always maximizing immediate utility? Would occasional manual practice or continued verification change the conclusions?
-
The simulations show that randomness can push workers from verification into full delegation after which they cannot recover. Could randomness also push workers toward verification or manual work and how would allowing these transitions affect the conclusions?
Thank you for the response and the revised manuscript. However, my main concerns remain. The core learning assumptions are unchanged in the revised version, and the relaxations discussed in the rebuttal such as plateaus and a positive default skill level are not analyzed. I therefore maintain my score.