acceptodds
Under review as a conference paper at ICLR 2027

STQA: A Dataset and Road-Subgraph-Augmented Language Model for Spatiotemporal Traffic Question Answering

Abstract

Spatiotemporal question-answering is crucial in transportation scenarios. However, existing traffic models typically focus on fixed numerical tasks, while general-purpose language models struggle to integrate temporal dynamics, directed road network dependencies, and heterogeneous contextual evidence. We investigate whether diverse traffic-analysis tasks can be unified through a natural-language interface while retaining structured spatiotemporal evidence. To this end, we introduce STQA, a dataset of approximately 120,000 question–answer pairs spanning ten subtasks across forecasting, imputation, classification, and explanation. STQA aligns traffic observations with road-network context to support both numerical analysis and contextual explanations of traffic patterns and event responses. We further propose RoadSubLM, a road-subgraph-augmented language model that uses a directed graph encoder to represent local road-network context as compact graph soft tokens. Auxiliary supervision reconstructs adjacency and road distances from graph-token hidden states, encouraging road-network structure to remain recoverable after graph–language integration. On STQA, RoadSubLM outperforms the evaluated language-model and time-series-language baselines across all ten subtasks, with particularly large gains in imputation, spatial-propagation classification, and contextual explanation. The dataset and code are available at https://anonymous.4open.science/r/STQA-66F3.

Primary area:
datasets and benchmarks
Keywords:
Traffic Question Answering, Spatiotemporal Dataset, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_UckWOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Existing traffic models mainly focus on numerical forecasting or classification for individual tasks, while general-purpose LLMs struggle to integrate traffic dynamics, directed road networks, and heterogeneous contextual information. To address these limitations, this paper introduces STQA, a benchmark constructed from existing traffic and incident data, comprising approximately 120K question–answer pairs across four categories and ten subtasks. The paper also proposes RoadSubLM, which extracts local road subgraphs and employs a graph encoder to generate graph soft tokens for integration into an LLM. Structural reconstruction supervision and a multi-stage training strategy are further employed to enhance the utilization of graph information.

Critical strengths

1.The paper introduces STQA, which integrates forecasting, imputation, classification, and explanation tasks into a unified question-answering framework. This broad task coverage is valuable for exploring the potential of natural-language interaction in intelligent transportation systems.

2.The paper evaluates RoadSubLM across ten tasks and compares it with several language-model baselines, including proprietary models and fine-tuned open-source models. In addition, ablation studies and further analyses are provided to investigate the contributions of different components.

Critical weaknesses

1.Although a unified natural-language interface is valuable for traffic analysis, is it really necessary to perform all underlying tasks, particularly forecasting and imputation, through autoregressive language generation? What advantages does the end-to-end design of RoadSubLM offer over an alternative framework in which an LLM interprets the question and selects the appropriate task, specialized spatiotemporal models perform the underlying computations, and the results are subsequently presented in natural language? Furthermore, RoadSubLM primarily integrates existing techniques, including graph attention mechanisms, graph soft tokens, structural reconstruction objectives, LoRA-based fine-tuning, and GRPO.

2.The paper presents the STQA benchmark as one of its primary contributions. However, the accompanying repository provides only the model implementation and data format specifications, without releasing the actual dataset. This makes it difficult to independently examine the quality of the generated question–answer pairs, task distributions, annotation procedures, and data splits.

Questions for authors

1.The paper states that the training, validation, and test sets use non-overlapping sensor groups and time ranges. However, each QA sample also incorporates neighboring sensors through a local road subgraph. Could the authors clarify whether the subgraphs of test samples may contain sensors that were used during training? Additionally, how are overlapping temporal windows and multiple QA samples associated with the same traffic incident handled to prevent potential data leakage across splits?

2.In Section 3.2, the paper employs a graph encoder to compress road-network information into graph soft tokens. However, the w/o graph module ablation in Table 2 does not seem sufficient to determine whether the performance improvements stem from the graph encoding mechanism itself or simply from providing additional neighboring information. Could the authors clarify how these two factors can be distinguished?

2 comments

W1: End-to-End Design and the Scope of Novelty.

Thank you for the question. This paper asks whether a shared graph–language model can perform forecasting, imputation, classification, and explanation across ten tasks under a unified STQA protocol. Our contributions include an STQA QA benchmark that organizes different answer types around the same traffic evidence; compression of directed road subgraphs and neighboring observations into graph soft tokens usable by a language model; retention of road connectivity through structural reconstruction constraints; and training of unified answering ability with task rewards. This design can produce numerical or categorical results and corresponding explanations from the same spatiotemporal context within a single QA interaction. The following example shows an explanation provided after a model prediction, illustrating how the model understands traffic spatiotemporal information and the basis of its prediction. In addition to numerical prediction and classification, our model can provide explanations and analysis, making its outputs interpretable and allowing it to answer spatiotemporal questions more comprehensively, rather than calling a separate model for each task and incurring complex training and very high time costs.

Case:The target flow on 718283 falls from 100 to 79 veh/h over the 15 measured steps because the current time is 2023-12-24 21:50, satisfying a weekend period (Sunday 21:00), indicates that reduced weekend commuting and night entertainment travel are active, and these factors change usual demand on the corridor, thereby affecting flow. This same reduced demand explains why the target occupancy ratio falls from 0.05 to 0.04 while the target speed rises from 64.2 to 66.0 mph, as lighter traffic allows higher speeds and lower occupancy without indicating a measured jam.

W2: Dataset Availability.

Thank you for pointing this out. We have uploaded the code used for the experiments. The paper is currently under review; we will release the dataset after the review process and add the corresponding link to the text.

Q1: Subgraphs, Temporal Windows, and Event Leakage.

Our data split prevents leakage. We first split training, validation, and test examples by anchor time, then check their complete history and prediction windows for cross-split overlap. Traffic events occurring in the training period are not included in the validation or test sets. Event-task labels are determined only by new incident onsets within the prediction window; incidents that began earlier are not treated as new future events, and event descriptions are not used as inputs. The model's subgraph module receives traffic observations only from past time windows, so it does not cause data leakage either.

Q2: Graph Encoding or Additional Information.

Thank you for the question about the ablation study. To distinguish the contribution of graph encoding from that of neighboring information itself, we will add an information-matched control: we will retain the neighboring nodes, historical observations, and directed road-network relations selected by RoadSubLM for each example, remove graph encoding and soft tokens, and instead feed this information to the same language-model backbone as text. We will evaluate under the same data split and a comparable training budget, comparing "text input with the same information" against "graph-encoded input." This experiment is still in progress; we will report its results and exact configuration in a subsequent response.

Dear Reviewer UckW,

As the discussion period will conclude in a few days, we would greatly appreciate your timely feedback on the clarifications provided in our response. We hope that our explanations have adequately addressed your concerns, and we would be grateful if you could consider adjusting the score. Should you have any further questions or suggestions regarding the paper or our rebuttal, please feel free to share them. We would be happy to provide further clarification and make additional improvements to the manuscript.

Thank you again for your time and thoughtful consideration of our work. We look forward to your response.

Best regards, Authors.

Reviewer_nLihOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces STQA, a dataset of approximately 120,000 question–answer pairs for road traffic analysis. It covers ten subtasks across forecasting, imputation, classification, and explanation, combining traffic observations with road connections, weather, incidents, and road attributes.

The authors also propose RoadSubLM, which adds a graph module to Qwen3-8B. This module encodes neighboring roads’ traffic data and connections into compact tokens for the language model. Auxiliary tasks predict road connectivity and distances to help preserve structural information during training.

The authors evaluate the model against general-purpose language models and a time-series language model on STQA. They report improvements over the selected baselines, particularly in imputation, congestion propagation, and explanation. Validation also includes component ablations, comparisons across training stages, probing experiments, and inference-efficiency measurements.

Critical strengths

1.A useful dataset contribution. STQA provides approximately 120,000 question–answer pairs across ten traffic-analysis subtasks, combining traffic observations with road-network and contextual information. This enables evaluation of multiple traffic tasks within a shared natural-language interface.

2.Evidence that road-network information improves performance. The ablation results show substantial benefits on some tasks: congestion-propagation accuracy increases from 69.46% without the graph module to 80.29% with the full model. This supports the practical value of incorporating neighboring-road information into traffic question answering.

Critical weaknesses

1.Hyperparameter selection is unexplained. The paper reports loss and reward weights without explaining their selection or providing sensitivity analysis, leaving their influence on performance unclear.

2.Ablation results are insufficiently analyzed. The full model underperforms simplified variants or earlier training stages on several metrics. The limited discussion does not adequately justify the final configuration.

3.Explanation evaluation is insufficient. ROUGE-L measures similarity to constructed, DeepSeek-rewritten references, but does not adequately establish factual correctness or evidential support.

4.Loss definitions are inconsistent. Figure 2 and the main text use different loss names and weighting expressions without explaining their correspondence, creating ambiguity about the training objective.

Questions for authors

1.How were the loss and reward weights selected? Were they tuned on the validation set, and over what candidate values?

2.What criteria were used to manually verify the reference explanations? Was explanation correctness assessed independently of ROUGE-L?

3 comments

W1, Q1: Weight Selection and Sensitivity.

Thank you for the question. The weights in graph–language alignment balance answer learning and structural reconstruction; the three GRPO coefficients control format, task-answer, and graph-structure rewards, respectively. The values currently reported in the main text are λa=2, λr=0.005 and 0.5, 1, 0.1. We are checking the actual training configuration, candidate values, and validation-set selection records, and adding a weight-sensitivity experiment under fixed data splits and training budgets. These experiments are still in progress; we will subsequently provide the candidate values and corresponding model results..

W2: Ablation Results and Stage-Wise Regressions.

We will add analysis of the ablation results. Table 2 shows that 13 of the 17 metrics improve from graph–language alignment to GRPO, including numerical prediction, incident detection, and all three explanation metrics, while some R2, S1, AF1, and C2 metrics decline. Joint optimization of ten tasks involves trade-offs, and it is difficult to achieve the best result on many metrics across many tasks simultaneously. For example, the baseline ChatTime performs well on prediction tasks but poorly on explanation tasks. We selected the final configuration based on overall cross-task performance and task coverage. Considering the model's multitask performance, reinforcement learning improves R1, R3, C1, and the explanation tasks; in particular, C2 F1 improves by 11.49% after reinforcement learning, while the largest regression on other tasks is only 2.03%, which we consider acceptable.

W4: Loss Names in Figure 2 and the Main Text.

Thank you for pointing out this inconsistency. L_QA in Figure 2 and L_ans in the main text denote the same answer-generation loss, while λ_rec and λ_r in Equation (7) denote the same reconstruction-loss weight. We also confirm that Figure 2 omits the coefficient on the answer loss, so the current figure does not fully express the objective in Equation (7). We will revise the objective in the figure to L_align = λ_a L_ans + λ_r (L_adj + L_dist) and unify the notation throughout.

W3, Q2: Explanation References and Independent Evaluation.

Explanation references use traffic observations, events, time windows, and road attributes to identify candidate factors. Experts select direction-consistent factors and permitted claims before polishing. An independent Qwen3.8 Max audit checks all QA pairs for consistency, relevance, evidence, and completeness (95.27% pass); flagged items are manually reviewed and revised.

ROUGE-L measures ordered overlap with references structured as context → factor → causal pathway → observed change; it cannot alone verify factual correctness. We therefore add BLEU for text overlap and reference-factor coverage (at least one; all). These metrics and the cases below test whether answers use the supplied evidence. CaTS-Bench and PATRA also report ROUGE-L.

ModelT1 BLEUT2 BLEUT3 BLEUT1 ≥1 factorT2 ≥1 factorT3 ≥1 factorT1 all factorsT2 all factorsT3 all factors
RoadSubLM56.9243.2940.7291.39%71.96%77.42%68.37%51.94%49.03%
UrbanLLM51.6741.1934.4881.70%60.21%47.58%49.80%41.79%19.68%
TS-Reasoner47.7639.1015.0085.33%66.62%47.90%59.22%46.06%29.52%
ChatTime48.3241.1220.7585.33%68.76%51.45%58.28%47.80%34.52%
OLMo-340.9338.0430.9271.87%55.27%39.52%51.28%38.58%13.39%
Nemotron53.9041.0031.6380.89%60.08%39.52%57.47%41.66%13.87%
DeepSeek37.4441.0131.5474.02%58.48%42.26%49.93%40.72%14.19%

Case 1 Question: [information] At 2023-12-25 00:55, why do the target sensor’s flow and speed decrease over the observation window?

RoadSubLM: The current time is 2023-12-25 00:55, satisfying a late-night or early-morning period (Monday 00:00), i.e., a late-night low-demand period. This indicates that late-night low demand is active in the current window. Late-night conditions correspond to weaker base travel demand, thereby affecting flow.

Nemotron: Several schools are present nearby; the current time is 2023-12-25 00:55, satisfying a weekend period (Monday 00:00). This indicates that the school pickup/dropoff peak is active in the current window. School pickup/dropoff can change short-term travel demand during school commute periods, thereby affecting speed. Several schools are present nearby; the current time is 2023-12-25 00:55, satisfying a weekend period (Monday 00:00). This indicates that the school pickup/dropoff peak is active in the current window. School pickup/dropoff can change short-term travel demand during school commute periods, thereby affecting flow.

Evidence: RoadSubLM links Monday 00:55 to low night demand and lower flow. Nemotron calls Monday a weekend and invokes nearby schools and a school-run peak at 00:55, claims unsupported by the question.

Case 2 Question: [information] Why do the current one-hour speed and flow sequences at the target sensor differ from the reference time window one week earlier?

RoadSubLM: Current-window condition: dense fog is present in this window. Reference-window condition: no dense fog or directionally aligned glare condition is verified in this window. This indicates that dense fog is present in the current window but not in the reference window. Dense fog can reduce driver perception and reaction distance, thereby affecting flow.

DeepSeek: Current-window evidence: Monday 2023-12-05 06:55 (weekday). Reference-window evidence: Monday 2023-11-28 06:55 (weekday). This indicates that reduced weekend commuting is present in the reference window but not in the current window. Reduced weekend commuting changes the usual demand on commute corridors, thereby affecting flow.

Evidence: RoadSubLM traces the current-window fog through reduced visibility to flow. DeepSeek calls both dates weekdays yet invokes reduced weekend commuting and misses the fog contrast.

Case 3 Question: [information] Why is this road segment’s flow response smaller than average under similar light-rain events?

RoadSubLM: Under the same light rain condition, the current time is 2023-12-19 01:55, satisfying a late-night or early-morning period (Tuesday 01:00), i.e., a late-night low-demand period. This indicates that late-night low demand is active in the current window. Late-night conditions correspond to weaker base travel demand, thereby affecting flow.

TS-Reasoner: The road segment's flow response is smaller/weaker than the average level under similar light rainfall observed at the sensor location events because the current event-affected flow sequence shows a decrease after an upward spike, which is not observed in the average response to similar weather conditions.

Evidence: RoadSubLM links late-night demand to a weaker flow response under the same light rain. TS-Reasoner restates a fluctuation and assumes an unprovided average sequence without explaining the demand pathway.

Dear Reviewer nLih,

As the discussion period will conclude in a few days, we would greatly appreciate your timely feedback on the clarifications provided in our response. We hope that our explanations have adequately addressed your concerns, and we would be grateful if you could consider adjusting the score. Should you have any further questions or suggestions regarding the paper or our rebuttal, please feel free to share them. We would be happy to provide further clarification and make additional improvements to the manuscript.

Thank you again for your time and thoughtful consideration of our work. We look forward to your response.

Best regards, Authors.

Reviewer_UbaFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes STQA, a dataset of approximately 120,000 question-answer pairs for spatio-temporal traffic analysis. This dataset contains ten subtasks such as forecasting, imputation, classification, and explanation. The authors also propose RoadSubLM for selecting a local road subgraph and encoding neighboring traffic observations using a directed graph encoder. The obtained node representations are projected into graph soft tokens and input to an LLM. Two auxiliary objectives reconstruct directed adjacency and road distances from the hidden states in LLM. Experimental evaluation demonstrates that RoadSubLM achieves the best for all 17 reported metrics.

Critical strengths
  • A comprehensive spatio-temporal benchmark dataset with different traffic analysis tasks: (i) STQA includes not only conventional numerical spatio-temporal forecasting task but also event classification and contextual explanation for spatio-temporal question-answering. (ii) This dataset is equipped with road network connectivity, traffic observations, and heterogeneous contextual information.
  • Consistent improvements in extensive experiments: (i) RoadSubLM achieves the best results in all 17 metrics. (ii) The ablation and probing studies show that both graph integration and structural reconstruction contribute to performance.
Critical weaknesses
  • Insufficient identification of contribution of the graph representation: (i) It is not clear whether all the baselines can access the same neighboring observations and road network, or whether the authors run task-specific training.
  • The claimed contextual reasoning capability is not enough for the explanation evaluation: (i) The evaluation relies on ROUGE-L with reference answers. It is not clear how much the references are checked for factual accuracy, relevance, and completeness. (ii) STQA covers a single city and year with only 40 weather events.
Questions for authors
  • Could you clarify what neighboring traffic observations and road-network information all the baseline methods receive?
  • Could you compare the performance of RoadSubLM with the one of a simpler spatial encoding using the same neighboring information and training?
  • Could you provide human evaluation of the factual correctness and evidence grounding of the generated explanations?
  • Could you clarify how overlapping temporal windows and shared traffic events are handled across the train/validation/test splits?
4 comments

W1, Q1, Q2: Neighbor Inputs and the Contribution of Spatial Encoding.

Thank you for the question. We follow the principle of fairness in our experimental evaluation: all baseline models and RoadSubLM receive the same target node, three-hop neighborhood, historical traffic observations, and available road-network and contextual fields; the available information is the same. For Q2, we will replace graph attention with a simple GCN spatial encoder while holding the input neighbors, training data, language-model backbone, and training budget constant, directly testing the benefit of the encoding mechanism. The experiment is still running; we will report its results and exact configuration in a follow-up response.

W2: One City, One Year, and 40 Weather Events.

Thank you for the question. The 40 weather events refer to separately annotated extreme-weather events in the study area; they do not mean that the entire year contains only 40 weather observations. STQA also uses year-round, hourly, eight-variable AORC meteorological data aligned with 2,211 sensors, so everyday weather variation continuously enters question construction. A single extreme event may also span multiple locations and time windows. These 40 event records involve 142 reported locations and produce 1,299 distinct event–sensor pairs when matched within a 5 km radius, providing substantial diversity. Moreover, the Los Angeles region alone has extensive sensor coverage and traffic-event information: in 2023, 2,211 sensors covered approximately 6,897 square kilometers, with 100,805 traffic events, yielding rich combinations of traffic states, weather, and events.

Q4: Temporal Windows and Event Overlap.

Our data split prevents leakage. We first split training, validation, and test examples by anchor time, then check their complete history and prediction windows for cross-split overlap. Traffic events occurring in the training period are not included in the validation or test sets. Event-task labels are determined only by new incident onsets within the prediction window; incidents that began earlier are not treated as new future events, and event descriptions are not used as inputs.

W2, Q3: Explanation Evaluation and Reference Answers.

ROUGE-L measures ordered overlap with references structured as context → factor → causal pathway → observed change; it cannot alone verify factual correctness. We therefore add BLEU for text overlap and reference-factor coverage (at least one; all). These metrics and the cases below test whether answers use the supplied evidence. CaTS-Bench and PATRA also report ROUGE-L.

ModelT1 BLEUT2 BLEUT3 BLEUT1 ≥1 factorT2 ≥1 factorT3 ≥1 factorT1 all factorsT2 all factorsT3 all factors
RoadSubLM56.9243.2940.7291.39%71.96%77.42%68.37%51.94%49.03%
UrbanLLM51.6741.1934.4881.70%60.21%47.58%49.80%41.79%19.68%
TS-Reasoner47.7639.1015.0085.33%66.62%47.90%59.22%46.06%29.52%
ChatTime48.3241.1220.7585.33%68.76%51.45%58.28%47.80%34.52%
OLMo-340.9338.0430.9271.87%55.27%39.52%51.28%38.58%13.39%
Nemotron53.9041.0031.6380.89%60.08%39.52%57.47%41.66%13.87%
DeepSeek37.4441.0131.5474.02%58.48%42.26%49.93%40.72%14.19%

Case 1 Question: [information] At 2023-12-25 00:55, why do the target sensor’s flow and speed decrease over the observation window?

RoadSubLM: The current time is 2023-12-25 00:55, satisfying a late-night or early-morning period (Monday 00:00), i.e., a late-night low-demand period. This indicates that late-night low demand is active in the current window. Late-night conditions correspond to weaker base travel demand, thereby affecting flow.

Nemotron: Several schools are present nearby; the current time is 2023-12-25 00:55, satisfying a weekend period (Monday 00:00). This indicates that the school pickup/dropoff peak is active in the current window. School pickup/dropoff can change short-term travel demand during school commute periods, thereby affecting speed. Several schools are present nearby; the current time is 2023-12-25 00:55, satisfying a weekend period (Monday 00:00). This indicates that the school pickup/dropoff peak is active in the current window. School pickup/dropoff can change short-term travel demand during school commute periods, thereby affecting flow.

Evidence: RoadSubLM links Monday 00:55 to low night demand and lower flow. Nemotron calls Monday a weekend and invokes nearby schools and a school-run peak at 00:55, claims unsupported by the question.

Case 2 Question: [information] Why do the current one-hour speed and flow sequences at the target sensor differ from the reference time window one week earlier?

RoadSubLM: Current-window condition: dense fog is present in this window. Reference-window condition: no dense fog or directionally aligned glare condition is verified in this window. This indicates that dense fog is present in the current window but not in the reference window. Dense fog can reduce driver perception and reaction distance, thereby affecting flow.

DeepSeek: Current-window evidence: Monday 2023-12-05 06:55 (weekday). Reference-window evidence: Monday 2023-11-28 06:55 (weekday). This indicates that reduced weekend commuting is present in the reference window but not in the current window. Reduced weekend commuting changes the usual demand on commute corridors, thereby affecting flow.

Evidence: RoadSubLM traces the current-window fog through reduced visibility to flow. DeepSeek calls both dates weekdays yet invokes reduced weekend commuting and misses the fog contrast.

Case 3 Question: [information] Why is this road segment’s flow response smaller than average under similar light-rain events?

RoadSubLM: Under the same light rain condition, the current time is 2023-12-19 01:55, satisfying a late-night or early-morning period (Tuesday 01:00), i.e., a late-night low-demand period. This indicates that late-night low demand is active in the current window. Late-night conditions correspond to weaker base travel demand, thereby affecting flow.

TS-Reasoner: The road segment's flow response is smaller/weaker than the average level under similar light rainfall observed at the sensor location events because the current event-affected flow sequence shows a decrease after an upward spike, which is not observed in the average response to similar weather conditions.

Evidence: RoadSubLM links late-night demand to a weaker flow response under the same light rain. TS-Reasoner restates a fluctuation and assumes an unprovided average sequence without explaining the demand pathway.

Dear Reviewer UbaF,

As the discussion period will conclude in a few days, we would greatly appreciate your timely feedback on the clarifications provided in our response. We hope that our explanations have adequately addressed your concerns, and we would be grateful if you could consider adjusting the score. Should you have any further questions or suggestions regarding the paper or our rebuttal, please feel free to share them. We would be happy to provide further clarification and make additional improvements to the manuscript.

Thank you again for your time and thoughtful consideration of our work. We look forward to your response.

Best regards, Authors.

Reviewer_UbaFOpenReview ↗

Response to Author Response

Thank you for the clarification. I keep my score.

Reviewer_61qVOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper introduces STQA, a multi-task spatiotemporal traffic question-answering dataset, and RoadSubLM, a graph-augmented large language model designed to answer natural-language questions about traffic conditions. STQA contains approximately 120K question–answer pairs spanning ten subtasks across traffic forecasting, imputation, classification, and contextual explanation. The dataset integrates traffic observations with road-network topology, weather, incident records, and geographic information.

RoadSubLM incorporates local upstream and downstream traffic information through a directed graph-attention encoder, which projects neighboring sensor representations into graph soft tokens for integration with a Qwen3-8B backbone. The model additionally employs structural reconstruction objectives to preserve road connectivity and distance information, alongside a three-stage training strategy involving supervised fine-tuning, graph–language alignment, and task-aware reinforcement learning using GRPO.

Experiments are conducted on the proposed STQA benchmark against eight LLM-based baselines. RoadSubLM achieves the best reported performance across all 17 evaluation metrics, with notable improvements in flow imputation, congestion propagation prediction, and contextual explanation generation.

Critical strengths
  1. The introduction of STQA, comprising approximately 120K question–answer pairs across ten subtasks, is a meaningful contribution toward unifying traffic forecasting, imputation, classification, and contextual explanation within a natural-language interface. Its integration of traffic observations, road-network topology, weather, and incident information provides a potentially valuable benchmark for advancing spatiotemporal reasoning research.

  2. RoadSubLM introduces a practical framework for incorporating directed neighborhood dependencies through compact graph soft tokens and auxiliary structural reconstruction objectives. The reported improvements across all 17 evaluation metrics, particularly in flow imputation and congestion propagation prediction, are supported by ablation studies demonstrating the contribution of graph augmentation. This provides encouraging evidence that explicitly modeling spatial relationships can improve LLM-based traffic analysis.

Critical weaknesses
  1. The experimental evaluation considers a small selection of general-purpose and time-series LLMs, while overlooking several closely related spatiotemporal question-answering and reasoning approaches (UrbanLLM, STReason, and TSReasoner). Furthermore, strong LLM-based traffic forecasting and imputation methods are not considered. This limits the comprehensiveness of the evaluation.

  2. The three explanation subtasks are evaluated exclusively using ROUGE-L against partially LLM-generated reference answers. Lexical similarity does not adequately measure factual correctness, evidence grounding, causal validity, or the quality of spatiotemporal reasoning. Moreover, no human evaluation or independent factual-consistency assessment is provided. Given that contextual reasoning is a central motivation of the paper, this substantially weakens the evidence supporting its reasoning capabilities.

  3. The proposed model combines graph augmentation, structural reconstruction, and task-aware reinforcement learning, while the baselines do not clearly receive comparable structural information or training opportunities. Although component-level ablations are provided, they do not fully represent the effects of architectural innovations from differences in input information and optimization. Therefore, it remains difficult to determine whether the reported improvements arise from the proposed methodological contributions or from unequal experimental conditions.

  4. The STQA dataset is constructed exclusively from Los Angeles County traffic data collected in 2023. Despite incorporating multiple sensors and task categories, the evaluation does not investigate cross-city, across time-periods, or out-of-distribution generalization. Traffic patterns, road-network structures, weather conditions, and incident characteristics can vary substantially across regions and periods. Therefore, the results provide limited evidence that the proposed framework generalizes beyond the specific dataset and environment used for training and evaluation.

  5. A substantial portion of STQA is constructed through automated task formulation and LLM-assisted language generation. Although the authors state that generated outputs were manually reviewed, the paper does not provide quantitative annotation-quality assessments, inter-annotator agreement, or systematic evaluations of question diversity and answer correctness. Furthermore, explanation references are derived from predefined factors rather than independently validated reasoning.

Questions for authors
  1. How does the model compare to spatiotemporal QA and reasoning approaches (e.g., UrbanLLM, STReason, TSReasoner) and LLM-based traffic prediction methods (e.g., ST-LLM+, TimeCMA) ?

  2. How do the authors verify that generated explanations are factually correct and grounded in traffic observations rather than merely matching reference text?

  3. Do all baselines receive equivalent traffic, spatial, and contextual information?

  4. Has RoadSubLM been evaluated on unseen cities, road networks, or time periods?

  5. What proportion of the approximately 120K QA pairs was manually reviewed, and what criteria were used?

4 comments

W1, Q1: Coverage of Relevant Baselines.

Thank you for suggesting the inclusion of additional related methods. We have added UrbanLLM and TS-Reasoner under the same STQA training and evaluation setup as the existing baselines; as shown below, RoadSubLM achieves better results on the reported metrics across all ten tasks. ST-LLM+ and TimeCMA primarily target numerical sequence prediction. Their native outputs cannot directly cover incident detection, disruption classification, and natural-language explanation in this benchmark. Designing new task heads, label mappings, and text generators for these tasks would compare newly constructed extended systems rather than the original methods. We therefore distinguish them from full-task comparisons among unified QA models. RoadSubLM aims to handle numerical, classification, and explanation questions with one model and to incorporate spatiotemporal evidence into its answers, improving its applicability to dynamic and interactive analysis and enabling users to interact with spatiotemporal traffic data in natural language.

TaskMetricUrbanLLMTS-ReasonerRoadSubLM
R1MAE / RMSE25.68 / 34.0923.34 / 28.8221.72 / 25.96
R2MAE / RMSE3.28 / 6.353.06 / 5.762.27 / 2.77
R3MAE / RMSE13.27 / 15.9712.58 / 15.158.05 / 9.67
AF1Acc. / F150.84 / 42.0154.49 / 42.0257.02 / 56.63
C1Acc. / F149.78 / 33.2356.40 / 55.3060.09 / 56.33
C2Acc. / F133.60 / 21.2138.18 / 33.3339.69 / 33.67
S1Acc. / F150.22 / 40.8964.98 / 62.0180.29 / 79.80
T1ROUGE-L60.2562.7170.55
T2ROUGE-L58.3960.2467.09
T3ROUGE-L36.3544.6952.69

W3, Q3: Baseline Inputs and the Graph Mechanism.

We follow the principle of fairness in our experimental evaluation: all baseline models and RoadSubLM receive the same target node, three-hop neighborhood, historical traffic observations, and available road-network and contextual fields; the available information is the same.

W4, Q4: Generalization Across Cities, Road Networks, and Time.

We added a small XTraffic test set covering seven QA tasks, with 100 examples per task and 700 examples in total, sampled from March, June, September, and December 2024. The results are shown below. RoadSubLM obtains an MAE of 21.55 veh/h on R1 flow forecasting, an MAE of 1.87 mph on R2 speed forecasting, and 80.0% accuracy on S1 congestion-propagation classification, maintaining strong performance on new spatiotemporal data.

ModelR1 MAER2 MAER3 MAEC1 ACC (%)C2 ACC (%)AF1 ACC (%)S1 ACC (%)
RoadSubLM21.551.8712.1566.039.061.080.0
UrbanLLM23.571.9521.5150.030.053.050.0
TS-Reasoner21.831.8915.1263.036.055.074.0
Chattime22.681.8538.6553.034.049.065.0
DeepSeek23.791.9216.3750.041.052.055.0
Nemotron23.611.9914.9251.037.052.050.0
OLMo-328.831.9820.4050.040.050.051.0

W5, Q5: Data Quality and Manual Review.

Thank you for your interest in the quality of automatically constructed QA pairs. Explanation answers are not generated freely by a language model alone: we first screen candidate factors using traffic observations, event labels, time windows, and static road attributes; human experts then select the factor most consistent with the direction of change and determine what may be included in the answer, followed by linguistic polishing. For internal quality control, we use Qwen3.8 Max, which was not used to create the data, as an LLM-as-Judge to inspect all QA pairs for information consistency, whether the question is answered, evidence presence, and data completeness. The recorded automatic-check pass rate is currently 95.27%; the remaining flagged examples are then manually reviewed and revised to ensure data validity and reasonableness.

W2, Q2: Factual Correctness and Evidence in Explanations.

ROUGE-L measures ordered overlap with references structured as context → factor → causal pathway → observed change; it cannot alone verify factual correctness. We therefore add BLEU for text overlap and reference-factor coverage (at least one; all). These metrics and the cases below test whether answers use the supplied evidence. CaTS-Bench and PATRA also report ROUGE-L.

ModelT1 BLEUT2 BLEUT3 BLEUT1 ≥1 factorT2 ≥1 factorT3 ≥1 factorT1 all factorsT2 all factorsT3 all factors
RoadSubLM56.9243.2940.7291.39%71.96%77.42%68.37%51.94%49.03%
UrbanLLM51.6741.1934.4881.70%60.21%47.58%49.80%41.79%19.68%
TS-Reasoner47.7639.1015.0085.33%66.62%47.90%59.22%46.06%29.52%
ChatTime48.3241.1220.7585.33%68.76%51.45%58.28%47.80%34.52%
OLMo-340.9338.0430.9271.87%55.27%39.52%51.28%38.58%13.39%
Nemotron53.9041.0031.6380.89%60.08%39.52%57.47%41.66%13.87%
DeepSeek37.4441.0131.5474.02%58.48%42.26%49.93%40.72%14.19%

Case 1 Question: [information] At 2023-12-25 00:55, why do the target sensor’s flow and speed decrease over the observation window?

RoadSubLM: The current time is 2023-12-25 00:55, satisfying a late-night or early-morning period (Monday 00:00), i.e., a late-night low-demand period. This indicates that late-night low demand is active in the current window. Late-night conditions correspond to weaker base travel demand, thereby affecting flow.

Nemotron: Several schools are present nearby; the current time is 2023-12-25 00:55, satisfying a weekend period (Monday 00:00). This indicates that the school pickup/dropoff peak is active in the current window. School pickup/dropoff can change short-term travel demand during school commute periods, thereby affecting speed. Several schools are present nearby; the current time is 2023-12-25 00:55, satisfying a weekend period (Monday 00:00). This indicates that the school pickup/dropoff peak is active in the current window. School pickup/dropoff can change short-term travel demand during school commute periods, thereby affecting flow.

Evidence: RoadSubLM links Monday 00:55 to low night demand and lower flow. Nemotron calls Monday a weekend and invokes nearby schools and a school-run peak at 00:55, claims unsupported by the question.

Case 2 Question: [information] Why do the current one-hour speed and flow sequences at the target sensor differ from the reference time window one week earlier?

RoadSubLM: Current-window condition: dense fog is present in this window. Reference-window condition: no dense fog or directionally aligned glare condition is verified in this window. This indicates that dense fog is present in the current window but not in the reference window. Dense fog can reduce driver perception and reaction distance, thereby affecting flow.

DeepSeek: Current-window evidence: Monday 2023-12-05 06:55 (weekday). Reference-window evidence: Monday 2023-11-28 06:55 (weekday). This indicates that reduced weekend commuting is present in the reference window but not in the current window. Reduced weekend commuting changes the usual demand on commute corridors, thereby affecting flow.

Evidence: RoadSubLM traces the current-window fog through reduced visibility to flow. DeepSeek calls both dates weekdays yet invokes reduced weekend commuting and misses the fog contrast.

Case 3 Question: [information] Why is this road segment’s flow response smaller than average under similar light-rain events?

RoadSubLM: Under the same light rain condition, the current time is 2023-12-19 01:55, satisfying a late-night or early-morning period (Tuesday 01:00), i.e., a late-night low-demand period. This indicates that late-night low demand is active in the current window. Late-night conditions correspond to weaker base travel demand, thereby affecting flow.

TS-Reasoner: The road segment's flow response is smaller/weaker than the average level under similar light rainfall observed at the sensor location events because the current event-affected flow sequence shows a decrease after an upward spike, which is not observed in the average response to similar weather conditions.

Evidence: RoadSubLM links late-night demand to a weaker flow response under the same light rain. TS-Reasoner restates a fluctuation and assumes an unprovided average sequence without explaining the demand pathway.

Dear Reviewer 61qV,

As the discussion period will conclude in a few days, we would greatly appreciate your timely feedback on the clarifications provided in our response. We hope that our explanations have adequately addressed your concerns, and we would be grateful if you could consider adjusting the score. Should you have any further questions or suggestions regarding the paper or our rebuttal, please feel free to share them. We would be happy to provide further clarification and make additional improvements to the manuscript.

Thank you again for your time and thoughtful consideration of our work. We look forward to your response.

Best regards, Authors.

Thank you for the detailed rebuttal and additional experimental results. These additions strengthen the empirical evaluation but still there are several concerns.

  1. W1: The inclusion of UrbanLLM and TS-Reasoner substantially addresses my concern regarding comparisons with relevant spatiotemporal QA approaches. I understand the authors' justification for excluding ST-LLM+ and TimeCMA from full-task comparisons. Nevertheless, these methods remain relevant benchmarks for the numerical forecasting subtask, where task-specific comparisons would help establish the competitiveness of RoadSubLM against specialized approaches.

  2. W2: While the additional BLEU and reference-factor coverage metrics provide useful evidence, my primary concern remains insufficiently addressed. These metrics primarily measure lexical similarity and the presence of expected factors rather than factual correctness, evidence-grounded reasoning, or the validity of the inferred relationships.

Furthermore, the provided examples demonstrate plausible explanations but do not necessarily establish that the identified factors are responsible for the observed traffic changes. For instance, Case 3 attributes a smaller-than-average flow response to late-night demand without directly demonstrating how this factor explains the difference from the reference response.

Given that contextual spatiotemporal reasoning is a central motivation of the work, I believe stronger evidence of explanation faithfulness and factual correctness is necessary.

  1. W4: The additional evaluation on 2024 data provides good evidence of temporal robustness. However, the test set contains only 100 examples per task and excludes the three explanation subtasks. Moreover, the results do not establish generalization across different cities or road-network structures. RoadSubLM also does not consistently outperform all baselines in this evaluation, particularly for speed forecasting and disruption classification.

  2. W5 : The clarification regarding expert-guided factor selection is helpful. But, LLM-based quality assessment does not independently establish annotation correctness, particularly for contextual explanations.

I've increased my score to 2 given the additional experimental results , but overall my main concerns regarding the validity of contextual reasoning evaluation, insufficient isolation of methodological contributions, and limited generalizability remain partially addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.