Terminology clarification: I'm going to write this review using the
standard terminology of principal-agent mechanism design. In this
terminology both the principal and the agent are humans. There is
also an AI in the model, however, the AI is not strategic. I will not
refer to the AI as an agent.
This paper considers a model for human-AI collaboration where a
principal chooses the rates of assigning tasks to human-alone,
AI-alone, and AI-human-combined review. The human (a.k.a. agent) in
the AI-human combined review chooses how much effort to put into the
monitoring of the AI. Key parameters are the allowed level of human
effort b and the accuracy δ (note: large δ is more
accurate). In this model b and δ are constraints. The
principal and the (human) agent choose their strategies in Nash
equilibrium. The agent aims to minimize their effort, while still
ensuring that the quality threshold δ is met.
The paper considers a static model and a dynamic learning model. In
the static model the equilibrium concept between the principal and
agent is Nash. In the dynamic learning model, human labels, when the
human is more skilled than the AI, are used to improve the AI and the
AI's performance is monotonic in the human labels collected. The
equilibrium concept is Stackelberg. (The dynamic model has an oddity
that I will discuss more later which is that human labels stop being
helpful for training the AI when the AI alone exceeds the skill of the
human alone. This is odd because the human+AI collaboration is
considered always better than the AI alone which means the human
always has something to offer.)
A main analysis of the paper is to characterize the equilibria in the
b,δ plane by partitioning it to regions that are infeasible and
where certain modes of task solving are not used (e.g., no
collaboration, full collaboration, partial collaboration, human only).
They interpret aspects of this analysis as identifying a collaboration
paradox and an automation cliff.
I struggled to find meaning in their partition of the b,δ.
Unfortunately these figures first appear as Figure 2 and 3 with
essentially no discussion. (That discussion is left to Section 4 of
the paper.) First, usually when an optimization problem is solved
with constraints, those constraints are binding. Assuming the
constraint on the human budget b and the accuracy δ are
binding, then I'm not sure what else we care about. I find a
different perspective on the b,δ plane to be more
understandable. The principal should aim to minimize b and maximize
δ. That gives a Pareto frontier. For any linear tradeoff
between the cost of b and the benefit of δ the principal will
choose a vertex of the Pareto frontier. These vertices tell a very
different story as to what is relevant. In all of the figures the
only vertices are at the b=0 corner (which is AI only, by
definition) and the corner at the top of the full collaboration
region. Nothing else in the plane seems relevant to the principal.
So why is the paper discussing it in detail?
Their modeling assumptions have a few issues, and I would very much
like to know whether any of the result are robust to these issues.
Issue 1: I don't understand why in the human alone condition, the
human cannot adjust their effort to lower accuracy if the δ
bound is not tight. I expect "fixing" this issue does not change the
results of the model.
Issue 2: I don't understand why in the dynamic model that the
human-only data is only useful when the human is better than the AI.
Given that the human-AI team is always better than the AI alone, this
means that the human has complementary abilities/information, and this
means that the AI can still be trained with the human data to be
better. I don't know if "fixing" this issue changes the results of
the model.
Comments on exposition:
-
I recommend that you do your analysis of the static model in
Section 2 (not Section 4). Figure 2 unfortunately had no
discussion at the point it appeared in the paper and this created a
lot of confusion as to how I should interpret it.
-
I recommend referring to the static model as "the static model" not
"the core model". In this context "core" has no helpful meaning.
-
I don't know how to read Figure 3 and understand how current
performance is sacrificed for future outcomes. Again there is no
interpretation of Figure 3 where it occurs. I recommend changing
this.
-
In the machine learning literature, I am most used to δ
being error (small is good) not accuracy.
-
Can you give some justification for switching solution concepts
from Nash to Stackelberg in the static and dyamic models?
Why study interior points rather than only the Pareto frontier?
[My apologies for the slow response, I didn't realize the ICLR author-response phase was interactive. Thanks to the authors to alerting me to this.]
The authors' answer about Pareto frontier does not resolve my non-understanding of the paper. Let me spell this out: Suppose the principal is given constraints (b,δ) and this is in the interior of the feasible set. Then this principal can either:
a) Save money, and spend b′<b to obtain the exact same δ. I do not understand why we care how to spend more than we need to spend to achieve a given δ.
b) Improve the accuracy, and achieve δ′>δ at the exact same cost b. I do not understand why we care how to produce less accurate answers than we could for the exact same budget b.
Once you realize we always want to be on the Pareto frontier---and there is no reasonable reason we cannot be---the analysis of the paper is mostly not so relevant.