acceptodds
Under review as a conference paper at ICLR 2027

Towards Machine-Readable Protocols for Responsible AI Agents

Abstract

Agents are increasingly relying on machine readable protocols such as model context protocol, agent-to-agent, skills, or data cards, which give agents a standardised interface to specialised tools and datasets to complete complex tasks. However, these protocols primarily prescribe when and how a specific tool, dataset, or model should be used, but do not surface responsible AI considerations, such as potential misuses or limitations, i.e., when tools or datasets should not be used or where their use might require additional guardrails. We observe that agents tend to ignore such limitations or do not follow responsible AI best practices and as a result, performance can suffer. To address this, we investigate a straightforward intervention: augmenting machine readable protocols with responsible AI information. Across multiple base models, domains, and tasks across the machine learning pipeline, we find that responsible AI augmentation reduces the severity of failures, broadens the exploration of accuracy-fairness trade-offs and can improve performance, but does not consistently change behaviour across all tasks. We further show that task-specific responsible AI prompting does not resolve (and may increase) these inconsistencies. Our work highlights the importance of considering responsible AI practices in the design of agentic frameworks and argues that machine readable protocols are a promising target for community investment given their scalability, durability, standardisation and auditability.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agents, responsible AI, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_1Q7ZOpenReview ↗
Rating
2(+1)
Confidence
2(−1)
Informativeness
1
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The authors ask whether augmenting machine readable protocols with responsible AI (RAI) information---whether task-agnostic or task-specific---improves performance (accuracy, fairness, robustness, proportion of critical failures) of eight language models (both proprietary and open source) on four tasks (selecting a dataset, writing code to train a ML model on biased data, choosing an LM based on its system card, performance at using skills).

They claim that:

  • i. Sec. 4.1 and Fig. 2: Absent the RAI augmentation, LMs focus on performance at the detriment of RAI best practices;
  • ii. Sec. 4.2 and Fig. 3: RAI decreases the rate of critical failures in the dataset selection task, and model cards with fairness metric lead agents to explore more the accuracy-fairness tradeoff in the model selection task;
  • iii. Sec. 4.3 and Fig. 4: When selecting two datasets, RAI augmentation can slightly improve performance (measured as "micro precision", i.e., agreement with human labels, in one task, and recall in another);
  • iv. Secs. 4.4 and 4.5, Figs. 5 and 6: RAI mostly does not lead to appreciable differences in the model building task or in skill use.
Critical strengths
  • i. The paper studies an important (and perhaps neglected) problem, proposing a solution simple, not costly, and easy to implement.
  • ii. The study is broad (8 models, 4 tasks, 2 controls).
  • iii. Admirably, the paper discusses its negative results as clearly as its positive ones (Figs. 5 and 6).
Critical weaknesses

The positive results seem to be confounded, for the RAI augmentation provides information essential to the tasks, which would be absent otherwise. In other words, it seems that the LMs employ the relevant facts, and not the RAI information per se (to establish if this is the case, the authors could run a control where such facts are provided to the model outside of the RAI framework). For instance:

  • Dataset selection: an agent is asked to retrieve a "relevant dataset", defined as such if "there is a target match and a demographic match" (see the system prompt in Appendix B.1). Then, the RAI augmentation directly provides such information (e.g., in "data biases" one can read "Sourced exclusively from a single academic medical center in California, likely overrepresenting insured, English-speaking patients and underrepresenting racial, socioeconomic, and geographic diversity.", see Appendix B.3, Table 1), which is absent in the base description.
  • Model selection: RAI augments model cards by adding fairness metrics which would otherwise be absent, but succeeding at this task requires those fairness metrics (see Sec 3.2, subsec 3).
  • In the cases where RAI augmentation consists of only "higher level" recommendations, said augmentation does not help (and sometimes it even hinders performance, e.g., in skill use).
Questions for authors

What would happen if the RAI augmentation provides only high level descriptions? Vice versa, what would happen in model selection if fairness metrics are provided outside of the RAI framework?

3 comments
AuthorsOpenReview ↗

Author response

Thank you for your detailed assessment. Your main concern is that the positive results may come from the augmentation adding information that the tasks need. We agree this was not separated clearly enough. We have reordered the results and measured how much RAI information is already in the baseline (see our replies to the global response).

Dataset selection. We show that there is implicit information about the RAI quantities of interest in each task. This is a significant signal for model building and dataset selection (both anonymised and non-anonymised settings), as we demonstrate in our new recall experiments. The benefit of augmentation is not predicted by how recoverable the information is, which argues against it only supplying of missing facts.

We acknowledge that the relevance criteria include RAI aspects and have moved all results under 'Explicit RAI information improves RAI behaviours when partially recoverable'.

Please note that the BioCADDIE task refers to datasets, queries and scores that do not depend on RAI criteria. Despite this, relevant datasets were more successfully recalled with the RAI augmented Croissant files for 6/8 models (with precision trade-offs), and this finding is only observed with dataset-specific RAI information (as per permutations).

Model selection. We agree that the fairness metrics are needed to improve fairness at all. We now describe this as a case where the information cannot be recovered, since no fairness ranking signal survives correction. Without the metric, agents choose almost only by accuracy (α\alpha near 0). Our new arms separate the metric from the limitations text. Adding the metric raises α\alpha to between 0.28 and 0.94 and raises HV for 7 of 8 models. Adding limitations on top raises α\alpha for GPT 4o, Haiku and Llama, and lowers HV for 5 models. So what the augmentation contains matters. Shuffling the metric values lowers the fairness weight for all 8 models, so the values are used. The limitations text has a smaller effect that differs by model.

Q1 (only high level descriptions). As shown in our new probe experiments, the baseline setup includes implicit RAI information in multiple cases. Our new results also specifically separate the contribution of metrics to that of limitations. We also show that the RAI prompt ("choose responsibly, consider fairness") moves α\alpha only a little for some models and lowers HV for several. The skill augmentation is high level and gives no benefit. It lowers the pass rate for GPT 5 and Gemma 4. Concrete facts change behaviour where they are the bottleneck, and advice does not do so reliably. We now state this as a result.

Q2 (facts outside the RAI framework). In our setup the fairness metrics are part of the RAI information. They are provided in the 'metrics' section of the model card, not in a separate RAI section. The added metric only arm and the shuffled metric test show what the values do. The metric plus limitations arm shows what the extra text adds. See the global response for details.

Mixed results. As you note, our results are mixed. Explicit augmentation can be needed but is not enough. Prompting can give large gains (model building) and also harm (BioCADDIE recall, model selection). Skill limitations and prompts do not work well. We think this is an important result. It shows there is no simple fix for RAI behaviour, and that protocol design should take RAI into account.

We hope these experiments clarify our message and support a better assessment of soundness and contribution.

Thank you for engaging with the reviews and for all the additional experiments.

  • In model selection, the new experiment with only the fairness metric partially addresses my second question. The results seem to suggest that the metric accounts for most of the change in the agents' choices (whereas the limitations text has mixed effects, which depend on the model). This clarifies the role of the intervention, but it also supports my original concern that the improvements come from providing facts that the agent needs to solve the task.
  • I understand (and appreciate) your probe experiments, which could provide evidence against my concern just above. However, their interpretation remains unclear to me. For example, recall@1 does not measure whether the model recovers the facts needed in the original decision (e.g., the population of a dataset). Moreover, a nonsignificant correlation across 8 models does not exclude that the explanation for the improvement is the missing information.

Overall, I acknowledge your engagement, the additional experiments, and the improvements, and thus raise both the soundness and the rating to 2. All these changes, however, also decrease my confidence (in part because they are substantial and were provided the last day of the discussion) and do not resolve my main concern.

Finally, I encourage you to update the manuscript, so that the reframing is easier to follow.

Thank you for your feedback and for increasing your score in this short time frame. Unfortunately, running these experiments has been quite time-consuming and has prevented us from engaging sooner or providing a revised text.

  • We agree with your reading of the model selection results. The fairness metric accounts for most of the change in choices, and the limitations text has mixed effects that depend on the model. Where the information a task needs is not available to the agent, the improvement comes from supplying that information. We say this directly in our revised results. We see it as the main point for case (a), since current protocols typically leave this information out.

  • For the dataset selection task, we repeated the recall test using only the use case, limitation and bias fields, which carry the information the decisions depend on. Recall@1 was still significant at p<0.001p<0.001 for all models. This is closer to the facts the task needs, but it is still an indirect test, so we treat it as weak evidence. The direct fact test will follow in the revision. While our results do not exclude an effect of recoverability on the results (small n correlation), we also note that we found no evidence of such relationship despite estimating a high number of pair-wise correlations (stage 1 and stage 3, multiple recoverability metrics and multiple outcome metrics).

Revised Results section summary

4.1 Implicit RAI information is present in LLMs and agents to a variable extent. The recovery tests. Models identify anonymised datasets, write RAI fields from a dataset name and description, and predict the fairness metric of candidate models. It shows that dataset descriptions carry implicit RAI information, and that fairness rankings are hard to recover.

4.2 Explicit RAI information is necessary when it cannot be recovered. Model selection task. Compares the baseline, the fairness metric only, and the metric plus limitations, using hypervolume and the implied fairness weight α. It includes the permutation tests on the metrics and the limitations text, and shows that the metric drives most of the change.

4.3 Explicit RAI information improves RAI behaviours when partially recoverable. Dataset selection with anonymised and not anonymised descriptions (critical failures, precision), and whether recovery scores predict outcomes. The model building negative results are also included here.

4.4 Explicit RAI information can improve outcomes on non RAI tasks. BioCADDIE dataset selection task. Recall of relevant datasets with the RAI fields, the information tier ablation, and the permutation results.

4.5/6 RAI prompting and skills. Kept from the submitted version, covering prompt effects and the skills results.

Appendix. The hypervolume and α definitions, the permutation tables, and the RAI prompt arm for model selection.

Sections 4.3 and 4.4 are largely the existing results moved into the new order. The new experiments sit in 4.1 and 4.2, with the correlation analysis added to 4.1 and 4.3.

Reviewer_mSi1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies whether machine-readable protocols used by AI agents can be augmented with more information to improve agent behavior without modifying the underlying models. The proposed intervention adds information such as dataset limitations and biases, model fairness metrics and caveats, or responsible-use instructions to Croissant metadata, model cards exposed through MCP-like interfaces, and SKILL.md files. The authors evaluate eight language models on four stages of an ML workflow: dataset selection, model development on biased data, model selection, and skill use.

Critical strengths
  1. The paper tests eight models across different machine learning subtasks, including dataset selection, model building, model selection, and skill use.
Critical weaknesses
  1. The paper is overclaiming its contributions in its current form. The experiments are only completed on machine learning subtasks with very specific datasets, and they feel disconnected to “responsible AI”, which typically means safer, more calibrated and trustworthy AI. It would be hard to treat the results as solid evidence for the claims.
  2. It is unclear how the proposed intervention method differs from simply annotating more and better information for the artifacts/tools. It seems like the paper is not making a meaningful technical contribution as well.
Questions for authors

Please see weaknesses.

1 comment
AuthorsOpenReview ↗

Author response

Thank you for your review.

W1 (overclaiming, tasks not related to RAI). Our tasks include a range of best practices of Responsible AI, such as selecting datasets accounting for demographic representation, writing code for training machine learning models from biased datasets, and selecting models based on a trade-off between performance and fairness metrics. These are all critical aspects of RAI (representation, robustness, fairness). Our choice of isolating these tasks is to ensure that one failure mode does not dominate our findings. We further note that the skills task includes other RAI aspects, such as privacy or source verification.

W2 (how is this different from better annotation?). While we agree with the reviewer that artefacts/tools should be better documented, our argument is specific to machine readable protocols and their impact on agent behaviour. Some of our results are unexpected:

  • Explicit information changes behaviour even when implicit information is present.
  • How well a model recovers information does not predict its behaviour.
  • RAI information changes recall on BioCADDIE, whose queries have no RAI criterion.
  • What the augmentation contains changes behaviour.
  • RAI augmentation is often not enough, and RAI prompts can have mixed or negative effects (Sections 4.4 and 4.5).

Please see the global response for the added contextualisation in terms of RAI information recovery.

We hope this clarifies our contribution and experimental setup. We'd be happy to answer further questions and would appreciate an increase in your score if you are satisfied.

Reviewer_Fao7OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to augment existing agent communication protocols such as MCPs, dataset cards or skill markdown files with additional information that reflects responsible AI best practices. This way, the hope is that the AI agents will follow the injected responsible AI guidelines and improve performance. Specifically, the proposed RAI augmentation considers dataset cards, model cards that are typically surfaced in various MCPs, and SKILL.md files that are used to define new skills for a model. The empirical evaluation is carried out on several LLMs including closed models such as Claude, GPT and Gemini, as well as open models like Gemma and Llama. The results demonstrate that in some cases the proposed RAI augmentation helps the performance of the models improve. However, overall it seems that the additional RAI information is not sufficient to reliably alter the behaviour of the models/agents.

Critical strengths
  • The quality of the presentation is overall fairly good. The paper is well written and the content is relatively easy to follow.

  • The empirical evaluation is fairly extensive including a wide variety of models and datasets/tasks. The results are presented in a clear manner and it is fairly straightforward to understand the benefits of the proposed approach.

Critical weaknesses
  • My main concern is that the proposed approach lacks any sort of guarantees. Namely, there is no formal insight into how or when the models really use the RAI augmentation. We merely hope that the models won't ignore the extra fields added to the SKILL.md file for example.

  • Overall, I found the contribution fairly limited.

Questions for authors
  • I was wondering if fine-tuning some sort of RAI information would help the models follow the instructions better.
2 comments
AuthorsOpenReview ↗

Author response

Thank you for your comments and for noting the extent of our experiments and the clear presentation.

W1 (no guarantees, do models use the augmentation?). We agree that the current framing does not provide guarantees, which we explicitly mention in section 4.2 (model building task) and in our discussion. However, providing explicit and relevant RAI information is often a necessary condition for improvement. This result is illustrated across multiple experiments (sections 4.1 to 4.3). To answer the question 'when RAI information is used', we perform permutation experiments:

  • In BioCADDIE, swapping the RAI fields between datasets (same length and format) lowers recall of relevant datasets by up to 42% (Haiku 4.5).
  • In model selection (new), swapping the fairness metrics lowers the fairness weight α\alpha for all 8 models. Swapping the limitations text has a smaller effect that differs by model. Tables are in our replies to the global response.
  • In the skills task, the augmentation changes neither RAI behaviour nor the pass rate. This shows that adding text to skills is not enough, and that protocol design needs to take RAI into account.

Explicit RAI information is needed when it cannot be recovered (model selection) and beneficial under partial recovery (dataset selection). In other cases it is not enough.

W2 (limited contribution). Our contribution is an empirical study of how the content of protocols affects agent behaviour. Our work highlights how RAI augmentation can affect RAI behaviours and outcomes in agents. Some of our results are unexpected:

  • Explicit information changes behaviour even when implicit information is present.
  • How well a model recovers information does not predict its behaviour.
  • RAI information changes recall on BioCADDIE, whose queries have no RAI criterion.
  • What the augmentation contains changes behaviour.
  • RAI augmentation is often not enough, and RAI prompts can have mixed or negative effects (Sections 4.4 and 4.5).

These negative results require the explicit involvement of the community to design RAI-aware agent harnesses, protocols and prompts. We have rephrased different sections of the work to better express this intent and hope that the reviewer sees value in bringing these results and discussion forward.

Q1 (fine tuning). In the present case, we focus on interventions that can be performed across all LLMs and agents, including closed source. Finetuning could be a potential avenue for open-source models, although one could argue that RLHF and various post-training stages and guardrails do provide some form of RAI best practices finetuning.

To investigate this later aspect, we conduct multiple recall experiments for each task, showing that LLMs and agents partially recover limitations from a dataset description and there is weak signal in predicting fairness rankings. Nevertheless, models with better recall of RAI fields do not achieve better outcomes. Together, these results highlight how the implicit presence of RAI information is not sufficient to obtain RAI behaviours in agents (see global response for details).

Thank you for the clarifications and additional experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.