acceptodds
Under review as a conference paper at ICLR 2027

PathTraj: Prompt-Aware Two-Stage Hierarchical Trajectory Generation from Language

Abstract

Synthetic trajectory generation offers a practical means of obtaining human mobility data when access to real records is limited by privacy concerns, collection costs, and proprietary restrictions. However, existing language-conditioned methods typically map a global condition directly to trajectories. As a result, they often overlook intermediate travel requirements, degrade on long journeys and under ambiguous instructions, and fail to preserve fine-grained motion dynamics. To address these limitations, we propose PathTraj, a prompt-aware two-stage hierarchical framework that decomposes generation into route planning and trajectory synthesis. In the planning stage, PathTraj grounds the instruction in the road network to infer feasible origin and destination segments, then decodes a connected sequence of segments under graph constraints and converts it into continuous geometric guidance. In the synthesis stage, a diffusion model generates coordinate-level trajectories conditioned on the planned route and the instruction, while a Motion Representation Extractor derives local motion features from intermediate denoising states to guide realistic speed and acceleration dynamics. Experiments on two real-world taxi trajectory datasets show that PathTraj outperforms existing baselines in instruction faithfulness and motion realism, and maintains this advantage across travel distances and levels of instruction detail. The code is available at https://anonymous.4open.science/r/PathTraj-EC63 .

Primary area:
learning on time series and dynamical systems
Keywords:
Language conditioned, Trajectory Generation, Hierarchical Architecture

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ojF1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors present a trajectory generation framework PathTraj, which decomposes language-conditioned trajectory generation into road-level planning and coordinate-level synthesis. A frozen language model supports endpoint inference and an autoregressive, graph-constrained route decoder. A diffusion model then generates coordinates conditioned on the route, instruction, and motion representations extracted from intermediate noisy trajectories.

Critical strengths

S1: The bridge between language and trajectory generation is well motivated. To be specific, PathTraj converts instructions into an explicit, graph-connected road route before synthesizing coordinates. Such representation is a good attempt to address the challenge of maintaining spatial coherence under language conditioning.

S2: According to the experiment results, the performance of PathTraj is superior in comparison against baselines. The improvements appear in both cities and extend beyond endpoint compliance to speed and acceleration distributions.

Critical weaknesses

W1: The comparison against ControlTraj may not be fair. According to the experiment results, PathTraj outperforms ControlTraj on instruction compliance and spatial fidelity, but the two methods don't have comparable structural conditioning. Appendix C.1 states that ControlTraj's RoadMAE-derived road embeddings are replaced with text embeddings, but PathTraj receives explicit guidance from a planned route. This removes a central component of ControlTraj and leaves unclear how much of the improvement comes from the proposed design.

W2: The motion realism results depend on a temporal setup that is not clearly explained. The reductions in speed and acceleration JSD are substantial and support one of the main claims. However, every trip is resampled to 200 points, and the authors do not clearly explain how timestamps are assigned afterward or how duration is determined for generated trips. Since these choices directly affect both speed and acceleration, it is difficult to tell whether the improvements reflect better motion modeling or the temporal assumptions used in evaluation.

Questions for authors

Q1. Please explain why the road conditioning was removed rather than retained alongside the text. Could you provide a comparison that preserves this conditioning, or supplies both generators with the same predicted route, to help establish where the gains come from?

Q2. How do you handle time when resampling each trip to 200 points and generating new trajectories?

3 comments

Thank you for your review. We have sincerely studied the weaknesses and questions raised, and we address them in detail below

Q1: removal of structured conditions

Thank you for raising the concern about removing structured road conditioning. PathTraj targets prompt-conditioned trajectory generation, where all trip requirements are provided in natural language rather than by directly supplying structured inputs. We therefore use natural language as the common condition for all models.

Regarding the original Controltraj, it is a route-conditioned generator. ControlTraj receives a road-segment route directly and generates GPS points along that route, but it does not predict the next segment or solve the route-generation problem. RoadMAE provides geometric embeddings of these supplied road segments, so the subsequent generation mainly interpolates or refines points along a known route. PathTraj instead first predicts endpoints and a road-segment route from the instruction, then generates GPS points along it. We therefore replace ControlTraj’s RoadMAE embeddings with language embeddings. Feeding the route directly to ControlTraj without requiring it to predict the route, as in its original setting, would give ControlTraj an unfair advantage.

We have not conducted a comparison in which both models receive routes. This would require adding a route-prediction module to ControlTraj or feeding PathTraj’s predicted route into ControlTraj, thereby changing the task scope to trajectory interpolation given a fixed route. The current comparison evaluates complete trajectory generation systems and does not isolate road conditioning from the coordinate generator. The ablation in Appendix D.2, Table 11 evaluates the contributions of the three designed components and supports their effectiveness. Removing route planning, motion guidance, or diffusion degrades the corresponding spatial, semantic, or motion metrics. We will clarify task scope in revised version.

Q2: time handling

Thank you for questioning about time handling. We handle temporal information through two steps: temporal refinement and fixed-length resampling. The time intervals between adjacent GPS observations in the raw data are approximately 1–5 seconds, most commonly about 3 seconds. To reduce the effects of irregular sampling, we interpolate between adjacent observations at 1-second intervals. This retains each trip’s start time, end time, and total duration TT, as well as the order of movement.

We then resample each refined trip to N=200N=200 points. The normalized time position of point jj is uj=jN−1, j=0,…,N−1u_j=\frac{j}{N-1},\ j=0,\ldots,N-1, and its timestamp is tjresampled=t0+ujTt_j^{\mathrm{resampled}}=t_0+u_jT. If the refined trip has fewer than 200 points, we add points by interpolation; if it has more, we uniformly downsample it. This gives all trips a fixed input length while preserving their endpoints, temporal order, and total duration. During generation, the model outputs 200 trajectory points. Given the generated trip duration TT, we assign timestamps using the same mapping: tj=t0+j199Tt_j=t_0+\frac{j}{199}T, so adjacent generated points are separated by t=T199t=\frac{T}{199}.

This preserves trip-level timing information, including each trip’s start and end times and total duration. Upsampling trajectories with 200 or fewer points does not discard observed points, whereas downsampling longer trajectories may smooth short-term timing and speed variation. This smoothing concern primarily affects trajectories with more than 200 points. As shown in Table 1, these trajectories are not majority in both datasets, so the overall effect is limited. 

Table1

StatisticChengduXi’an
Mean points142.34178.47
Standard deviation81.9893.7
25th percentile82107
Median127160
75th percentile186233
Percentile of 200 points79.40th65.10th
Trajectories exceeding 200 points20.60%34.90%

We greatly appreciate Reviewer ojF1’s thoughtful feedback and constructive suggestions, which have been instrumental in helping us refine our research and improve the clarity of our presentation. We have made every effort to thoroughly address each of your questions and concerns in our rebuttal.

We would sincerely welcome any additional comments or suggestions from Reviewer ojF1. Your continued feedback is extremely valuable in further strengthening the quality and clarity of our work. Furthermore, if you find that our responses have adequately resolved your concerns, we would be truly grateful if you would consider raising your score based on our detailed responses and additional explanations.

Reviewer_ojF1OpenReview ↗

Response by Reviewer

I appreciate the effort made by the authors for addressing my concerns. For Q1, the authors explained why it was infeasible to conduct a comparison in which both models receive routes. For Q2, the authors provide additional details regarding time handling. My concerns are partially addressed.

Reviewer_7N4MOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PathTraj, a prompt-aware hierarchical framework for language-conditioned trajectory generation. PathTraj plans an explicit route on the road network and then synthesizes the trajectory under its guidance. Experiments on two real-world taxi trajectory datasets demonstrate that PathTraj outperforms existing baselines across varying travel distances and instruction clarity levels.

Critical strengths

1.The choice of intermediate representation is appropriate. A discrete, connected, and checkable sequence of road segments is used as an explicit route plan, rather than a latent vector or topological embedding; connectivity is guaranteed by the decoding construction, not by a post-hoc penalty. This makes the language grounding verifiable and intervenable.

2.Planning and generation are decoupled. Deterministic route planning locks down the global structure, while diffusion is only responsible for local coordinates and motion texture; route errors can be detected before generation, modules can be replaced, and motion guidance is adaptively updated along with the denoising process.

Critical weaknesses

1.The paper lacks an end-to-end baseline using the same Qwen3.5-9B encoder and a comparable diffusion backbone, but without explicit route planning. Without it, one cannot rule out that the advantage comes from the additional structured conditioning and larger effective capacity rather than from plan-then-generate itself.

2.Baseline fairness and route-level evidence are insufficient. It is not explained whether InsTraj was given comparable spatial conditions and a comparable tuning budget; meanwhile, route-level compliance is supported only by qualitative figures, lacking quantitative metrics such as average deviation from the planned route, segment coverage, and completion rate, and the route-planning ablation lacks decisive route-level degradation evidence.

Questions for authors

1.Could the authors clarify whether any existing ablation or control in the paper already isolates the effect of explicit route planning from the additional structured conditioning and capacity? If not, how do they argue that the gains are attributable to plan-then-generate rather than to the extra conditioning?

2.Could the authors clarify whether InsTraj's tuning budget and spatial conditioning are specified in the appendix, and whether any quantitative route-level results are already reported? If only qualitative figures are available, could they explain why these are sufficient to support the route-planning claim?

2 comments

Thank you for your review. We have sincerely studied the weaknesses and questions raised, and we address them in detail below

Q1,W1,W2: Ablation and Contribution of explicit route planning.

Thank you for your comment on the Ablation settings and contribution of explicit route planning.

All baseline models and ablation variants use the same Qwen3.5-9B encoder, trajectory instruction pairs, and diffusion training and sampling settings. Therefore, existing w/o route planner ablation can be considered as end-to-end baseline with the same encoder and diffusion backbone.

To further evaluate route planning, we conducted an additional ablation replacing the learned planner with shortest-path routing. The results in Table 1, support that the learned route planner preserves the spatial distribution better.

Table 1

VariantDensity↓Trip↓Length↓
Shortest-path routing0.0332-0.0071
PathTraj0.02390.16650.0036

Q2,W2: InsTraj’s configuration and quantitative route-level results.

Thank you for highlighting the need to clarify the baseline settings and route-level evaluation. Appendix C.1 describes InsTraj’s architecture and conditioning mechanism. In our implementation, InsTraj receives instruction prompts as its sole conditioning input, without separately supplied endpoint or route conditions. We trained and inferred every baselines with the same setting described in Appendix B, Table 8. Each model is trained for 200 epochs on two NVIDIA A100 GPUs. We will make these shared settings explicit in the baseline description.

Regarding Quantitative route-compliance, results are already reported in Appendix D.3, Table 12, which evaluates the final coordinate trajectories. The metric currently named Route distance measures average deviation from the intended route; we will rename it Average route deviation for clarity. PathTraj achieves the lowest Average route deviation, Progress MAE, Normalized MAE, and Endpoint error in both evaluated cities.

We further conducted detailed evaluation on route using 2,000 samples per prompt-clarity level. Completion rate is approximately 0.95 for Coarse and Medium prompts (Table 2) and above 0.99 for Fine, Detailed, and Comprehensive prompts (Table 3). PathTraj consistently outperforms the shortest-path decoding control, providing additional quantitative evidence for the route-planning component.

Table 2

StatisticcoarseMediumShortest path
Completion rate↑0.94950.9485-
fcr↑0.66860.5975-
density↓0.05200.05440.0911

Table 3

StatisticfinedetailedcomprehensiveShortest path 
endpoint accuracy↑0.99800.99800.9985-
Completion rate↑0.99450.99600.99551
Reference route match↑0.89890.91160.91110.6691
Segment F1↑0.9660.97250.97360.8748
Mean absolute route-length error (segments)↓0.3142 0.3198 0.2662 0.9841 
Next-segment accuracy↑0.90150.91350.91610.7038

We greatly appreciate Reviewer 7N4M’s thoughtful feedback and constructive suggestions, which have been instrumental in helping us refine our research and improve the clarity of our presentation. We have made every effort to thoroughly address each of your questions and concerns in our rebuttal.

We would sincerely welcome any additional comments or suggestions from Reviewer 7N4M. Your continued feedback is extremely valuable in further strengthening the quality and clarity of our work. Furthermore, if you find that our responses have adequately resolved your concerns, we would be truly grateful if you would consider raising your score based on our detailed responses and additional explanations.

Reviewer_4o2UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PathTraj presents a two-stage framework for generating natural-language-conditioned mobility trajectories. It first uses a pretrained language model to infer reachable origin and destination road segments, then constructs a connected road-level route under graph constraints. A diffusion model subsequently synthesizes GPS coordinates conditioned on the route and instruction, while a pretrained motion representation extractor provides speed-, acceleration-, and heading-related guidance during denoising. Experiments on taxi trajectory datasets from Chengdu and Xi’an compare PathTraj with DiffTraj, ControlTraj, and InsTraj. The reported results show improvements in instruction-faithfulness, spatial/trip distribution metrics, positional errors, and motion statistics; ablations indicate that route planning, motion guidance, and diffusion each contribute to performance.

Critical strengths
  1. The explicit route-planning stage provides an interpretable bridge between language instructions and coordinate generation, addressing a meaningful limitation of end-to-end conditioning.
  2. The evaluation covers instruction compliance, distributional fidelity, motion realism, distance/prompt variation, and component ablations on two real-world city datasets, giving a relatively broad empirical picture.
Critical weaknesses
  1. The evaluation protocol needs stronger transparency. The paper does not make the construction and validation of instruction–trajectory pairs, the reliability of instruction-compliance metrics, or the degree of human/LLM involvement sufficiently clear to establish that the reported gains reflect robust semantic faithfulness.
  2. Evaluation is limited to two Chinese taxi datasets and a small set of baselines. The paper provides little evidence of transfer to other cities, mobility modes, or independently authored prompts, limiting claims about generality.
  3. The endpoint inference and greedy route decoder appear to make largely deterministic decisions; the paper does not sufficiently analyze failure cases when instructions are ambiguous, endpoints are misgrounded, or a valid route cannot be decoded.
  4. Runtime, computational cost, and sensitivity to key design choices (including the motion-guidance activation threshold) are not adequately characterized.
Questions for authors
  1. How are prompts and their semantic labels generated and validated, and are the instruction-faithfulness metrics evaluated independently of the model or data-construction pipeline?
  2. How often do endpoint inference or route decoding fail or violate the intended route requirements, and how does PathTraj handle ambiguous prompts or unreachable destinations?
  3. What are the training/inference costs and results on additional cities or independently authored prompts?
2 comments

Thank you for your comment. We have sincerely studied the weaknesses and questions raised, and we address them in detail below

Q1,W1: Prompt construction and instruction-faithfulness.

Thank you for raising the need to clarify how instruction–trajectory pairs are constructed. Instruction–trajectory pair construction is fully automated, with no human intervention. The data construction pipeline in Appendix A.3 and A.4 can be summarised recorded as following:

  • extract structured attributes from trajectories and associated context
  • populate attributes into predefined text templates
  • paraphrase the populated text

Because LLM is only used for paraphrasing rather than inferring instructions from raw trajectories, the instruction semantics remain grounded in those attributes, reducing the risk of semantic drift.

For instruction-faithfulness evaluation, PRR, TIS and FCR (FCS+FCD/2) are used. Although PRR reuses the prompt attribute-extraction procedure. TIS uses a separately trained TrajCLIP scorer, while FCR compares endpoint road function with the requested functions. Thus, PRR measures consistency within the construction pipeline, whereas TIS and FCR independently assess semantic alignment and endpoint-function compliance. We will clarify these dependencies and avoid presenting the metrics as fully independent.

Q2,W3: Endpoint and route generation.

Thank you for asking us to quantify endpoint and route generation failures. We therefore add endpoint and route generation evaluation shown in Table 1-2. Across 2,000 samples per clarity, Fine, Detailed, and Comprehensive prompts achieve about 99.8% endpoint accuracy and 99.5% completion, while Coarse and Medium prompts achieve at least 58% FCR and 95% completion.

To assess adherence to intended route requirements, we compare PathTraj with shortest-path decoding using predicted endpoints in Table 2. Against shortest-path decoding, PathTraj demonstrates better requirements following ability shown in Table 1 and 2.

Table 1

StatisticfinedetailedcomprehensiveShortest path
endpoint acc.↑0.99800.99800.9985-
complete rate↑0.99450.99600.99551
Ref. route match↑0.89890.91160.91110.6691
Segment F1↑0.96600.97250.97360.8748
Transition F1↑0.95790.96570.96670.8444
MAE↓0.31420.31980.26620.9841
Next-segment accuracy↑0.90150.91350.91610.7038

Table 2

StatisticcoarseMediumShortest path
complete rate↑0.94950.9485-
fcr↑0.66860.5975-
density↓0.05200.05440.0911

For ambiguous prompts, endpoint inference uses available semantic cues and still predicts a valid endpoint pair. When a prompt is less informative, inference may be close to random selection among valid endpoint pairs. Reachability is guaranteed by constraint that destination candidates are restricted to segments reachable from that origin. The decoder considers only legal outgoing segments which ensures connected routes.

Q3,W2,W4: Computational cost and evaluation on an additional city, and design sensitivity.

Thank you for asking us to report computational cost and evaluate an additional city. Each model was trained for 200 epochs on two A100 GPUs with the same training and evaluation settings in Appendix B, Table 8. Although PathTraj adds route-planning overhead shown in Table 3, the target of trajectory generation is offline rather than time-critical, interactive navigation. Thus,latency is not a hard constraint.

Table 3

ModelRoute latency↓Coor. latency↓Total latency↓efficiency (traj/s)↑
PathTraj54.160.7114.88.71
ControlTraj—31.231.232.05
InsTraj—29.329.334.13
DiffTraj—30.530.532.79

Regarding performance in an additional city, Porto, as shown in Table 4, PathTraj achieves the best results across all six metrics. Notably, the modest FCR gain may be caused by the limited POI coverage in Porto. Many road segments are assigned the same urban-function category and less distinguishable.

Table 4

modelsdensity↓length↓trip↓fcr↑PRR↑TIS↑
PathTraj0.04460.00650.23380.76340.79220.4999
Controltraj0.07040.00990.24500.75330.70660.4767
Instraj0.07190.00910.24330.73910.70860.4756
Difftraj0.06820.00900.25270.76280.71620.4813

Regarding sensitivity of motion guidance activation. The best schedules range from 0.5 to 0.7, which reduce motion predictor calls while maintaining performance. This suggests that guidance during early denoising may be less useful, possibly because motion cues are not yet informative. Late activation reduces cost further but worsens performance, because too few steps are influenced by motion guidance.

Table 5

Fully Active ThresholdDensity↓Length↓Trip↓efficiency (s/traj)↓
0.00.02240.00380.15940.1214
0.50.02230.00320.15850.0931
0.60.02250.00320.15810.0915
0.70.02280.00350.15950.0902
0.80.02310.00380.16140.0886
0.90.02390.00360.16650.0874
1.00.02600.00430.17450.0857

We greatly appreciate Reviewer 4o2U’s thoughtful feedback and constructive suggestions, which have been instrumental in helping us refine our research and improve the clarity of our presentation. We have made every effort to thoroughly address each of your questions and concerns in our rebuttal.

We would sincerely welcome any additional comments or suggestions from Reviewer 4o2U. Your continued feedback is extremely valuable in further strengthening the quality and clarity of our work. Furthermore, if you find that our responses have adequately resolved your concerns, we would be truly grateful if you would consider raising your score based on our detailed responses and additional explanations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.