acceptodds
Under review as a conference paper at ICLR 2027

TRACE: A Traffic Regulation Agent with Compositional Evidence

Abstract

Traffic regulation compliance requires reasoning over temporal events, participant roles, thresholds, and exceptions across continuous driving processes. However, fixed regulatory specifications offer limited flexibility in handling context-dependent legal conditions, while LLM-based assessments with static context are constrained by the evidence provided upfront. We propose TRACE, a traffic regulation agent with compositional evidence that separates reusable evidence computation from regulation-specific reasoning. A stateful runtime maintains factual and temporal evidence, while Law Skills guide an event-centered workflow to acquire and compose the evidence needed for compliance assessment. To extend its regulatory coverage, TRACE supports constructing initial skills for new regulations and refining their evidence procedures through execution feedback within fixed legal and tool boundaries. TRACE achieves 96.67% strict accuracy on 751 real-world traffic regulation compliance cases. Starting from automatically initialized skills, evolution improves strict accuracy from 46.6% to 81.4% on held-out driving cases. These results demonstrate accurate traffic compliance assessment and the ability to improve regulation-specific evidence procedures through execution feedback.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, Traffic Regulation Compliance, Driving Behavior Assessment, Intelligent Transportation Systems

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wUtYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TRACE is a tool-augmented agent for assessing traffic-regulation compliance from structured driving observations. It separates stateful factual evidence computation from regulation-specific reasoning through Law Skills that guide event anchoring, participant binding, temporal queries, and evidence composition. It also introduces a constrained skill-evolution procedure that revises evidence-acquisition workflows using development feedback and validation-based selection while keeping legal resources and tool semantics fixed.

Critical strengths

LLM-Evidence receives the factual outputs collected by TRACE, excluding the generated reasoning, decisions, and procedural skill content, yet achieves 85.22% accuracy versus TRACE’s 96.67%. The Law Skill, temporal-evidence, and adaptive-querying ablations further support the value of the proposed workflow on this benchmark. These controls make the contribution more convincing than a comparison against static baselines with substantially less factual information alone.

The development, validation, and test splits are source-log-disjoint within each regulation, and test labels are excluded from updates and selection. The gain from 46.59% to 81.40% is accompanied by paired statistical analysis in Appendix C. This supports the practical value of improving reusable evidence procedures through execution feedback.

Critical weaknesses

A central motivation is that separating factual computation from legal reasoning makes new or revised regulations easier to support. However, the experiments primarily demonstrate improved assessment within a fixed 13-regulation corpus. They do not directly establish the effort saved over updating a rule-based monitor when a provision or exception changes. TRACE still relies on regulation-specific Law Skills, evidence profiles, and calibrations, whose construction and maintenance costs are not quantified. Consequently, the results support a capable configured evaluator more strongly than the broader adaptation advantage motivating the architecture. This limits my assessment of the contribution’s significance.

The single custom benchmark leaves external validity uncertain. All reported results use cases from the authors’ collected driving logs, with manually specified assessment intervals and a shared annotation and calibration protocol. Held-out logs provide a useful internal test, but do not establish transfer to an independent data source or observation pipeline. No evaluation on a public driving dataset or independently constructed benchmark is reported, and the reproducibility statement only indicates that the data can be released after de-identification. Constructing a task-specific dataset is reasonable, but the lack of independent validation makes it difficult to determine whether the high accuracy reflects broadly reusable capabilities or a close fit to this particular evaluation setting.

Questions for authors

For a new or revised regulation, which skills, evidence profiles, calibrations, and runtime components must change? Can a focused comparison with updating a rule-based monitor substantiate the claimed adaptation benefit, including the required manual effort?

What evidence supports transfer to an independently collected driving dataset without extensive retuning of skills or calibrations? Can the benchmark, annotation protocol, and necessary evaluation artifacts be made available to enable independent validation?

Reviewer_Lv6DOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TRACE, an LLM agent that checks recorded driving logs for traffic-regulation compliance. Per-regulation "Law Skills" guide the agent to query a fixed fact runtime for event-, participant- and time-specific evidence. A constrained evolution procedure revises these skills from execution feedback while keeping legal text, thresholds and tools frozen. On 751 real-world cases covering 13 regulations, TRACE reaches 96.67% Strict Accuracy, against 86.55% for a rule engine. Evolution improves automatically initialized skills from 46.6% to 81.4% on held-out cases.

Critical strengths

S1. Solid, auditable system design. TRACE cleanly separates regulation-neutral fact computation from regulation-specific reasoning, with an explicit boundary on what skills (and their evolution) may change.

Evidence handling is careful: validity and provenance are tracked per item, missing observations are not treated as absence, supporting/contradicting/uncertain evidence is kept apart, and applicability is reported separately from compliance. The traces in App. D show these mechanisms operating in practice, yielding findings that are traceable to specific queries, frames and participants.

S2. Convincing empirical validation within the stated scope.

On 751 real-world, triple-annotated cases, TRACE substantially outperforms a rule engine that uses the same thresholds (96.67% vs 86.55% Strict Accuracy; FPR 0.20% vs 1.23%). The evidence-matched LLM-Evidence control (85.22%) shows that the gain comes from skill-guided acquisition and composition rather than from factual access alone, and the component ablations are consistent with this.

Critical weaknesses
  1. The system-level contribution is not clearly delineated.

The general pattern (a tool-augmented agent with skill files, plus parent-plus-patches procedure search with validation selection) is established. As a systems paper, the work does not need algorithmic novelty, but it should state clearly which design decisions are new and why they matter. This is especially important relative to Yu et al. (2024). The atomic propositions, thresholds and rule-engine baseline come from that work, and the paper does not say whether the evidence runtime is reused, extended or newly built. The constrained editable scope (Table 14) is presented as a key element, but its effect is not analyzed. For example, the paper does not report how often proposed edits violate the scope, or what happens if thresholds become editable.

  1. Technical details about the evidence tools are unclear.

The evidence tools are the foundation of TRACE: every fact the agent reasons over comes from them. Yet the paper gives almost no technical detail about them. Neither the main text nor the appendix lists the tools, gives their number, inputs/outputs or implementation, explains how they were built (manually, generated, or reused from Yu et al., 2024), or says which regulations each was designed for.

This gap matters for interpreting the results. The tools appear closely tailored to the 13 evaluated regulations. For example, distance.object_calibrated_clearance_window takes exactly the constants of the IMR 44.1 formula (App. D.1, Step 5). The rule engine (86.55%) and the single-prompt LLM-Evidence (85.22%) already perform well on the same facts. A large share of the performance may therefore come from the tools rather than from the skill-guided agent, but readers cannot judge this.

Questions for authors
  1. Could you provide a complete specification of the evidence tools: the full list, the number of tools, and each tool’s inputs, outputs and computation?
  2. Which components of TRACE are new relative to Yu et al. (2024), and which are inherited?
  3. How often do proposed skill edits violate the editable scope, and how does evolution behave if the thresholds are also made editable?
1 comment

I thank the authors for the detailed response. The rebuttal clarifies which parts of TRACE are new and gives me stronger evidence for the value of the Law Skills. I maintain my rating of 3.

Reviewer_evCSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TRACE, an LLM-based agent for traffic regulation compliance assessment. It combines stateful evidence retrieval, manually specified Law Skills, and iterative tool-based reasoning. The authors further introduce a constrained skill evolution mechanism that revises evidence-acquisition procedures using execution feedback.

Critical strengths

The paper addresses a practically important problem and provides a reasonably well-engineered system for evidence-grounded traffic regulation assessment. The empirical results are promising, particularly the performance of the full system and the improvements obtained through skill evolution.

Critical weaknesses

1. Lack of methodological novelty

There is no new method proposed beyond equipping an existing tool-using agent with carefully engineered domain knowledge and query procedures. The framework is combining established components.

2. The motivation of skill evolution is unclear

Automatically evolved skills achieve 81.4% accuracy, while manually authored skills achieve 96.67. So why use skill evolution? This paper does not show that evolution reduces expert effort.

3. Limited evidence of generalization

The empirical result does not show that the proposed procedures generalize to unseen regulations.

Questions for authors

(1) What is the fundamental methodological novelty of Law Skill-guided reasoning?

(2) How would TRACE compare against a strong ReAct-style baseline with equivalent tools, regulatory knowledge, and query budgets?

(3) What practical advantages does automatic skill evolution offer over manually authored skills in terms of human effort, computational cost, and adaptation to unseen regulations?

(4) Given that domain experts are already required to specify legal conditions and thresholds, develop evidence tools, and provide annotated data, how much additional effort would it actually take to manually author a high-quality Law Skill?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.