- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes DriftTraj, a framework that reformulates trajectory recovery as map-conditioned video generation. Sparse observations are rendered as markers on OpenStreetMap tiles, and missing movement is recovered by generating video frames in which a red marker traces the inferred path. The generator is trained via Drifting Models, enabling one-forward-pass generation per video block without iterative diffusion sampling. A differentiable red-marker extractor provides explicit geometric supervision for marker localization, and a Truth Forcing mechanism re-renders markers on the original map between autoregressive steps to prevent appearance drift. The model is pretrained exclusively on Porto and transferred zero-shot to Beijing, Xi'an, and Chengdu, with optional limited-data fine-tuning. Experiments across four cities and nine baselines support visual maps as a shared cross-city representation for trajectory recovery.
Critical strengths
-
Novel and well-motivated problem formulation. The central insight — that visual maps provide a city-agnostic shared representation, bypassing the road-segment vocabulary bottleneck of graph-based methods — is both original and compelling. The analogy to how human experts trace paths on maps is intuitive and well-articulated. This reformulation opens a genuinely new design space for trajectory recovery and related mobility tasks.
-
The differentiable marker-aware geometric supervision is a technically elegant solution to a real problem. The observation that RGB reconstruction loss alone may reproduce the static map background without accurately localizing the small moving marker is well-identified. The red-response extractor with its position-dependent centroid gradient provides a principled, differentiable path from pixel-level generation to explicit location supervision, avoiding the need for discrete marker detection that would break gradient flow. The ablation confirming that removing this loss causes early error spikes and instability validates its necessity.
-
The efficiency argument is well-supported and practically significant. The 198.8x speedup over a video diffusion reference, with 10,000 trajectories processed in 11.07 minutes, demonstrates that one-step generation makes vision-based trajectory recovery practicable rather than merely theoretical. This is an important feasibility result for the broader visual trajectory research program.
-
Comprehensive experimental design with honest framing. The four-city evaluation with zero-shot transfer and limited-data fine-tuning covers the key transfer scenarios. The paper is commendably transparent that the Top@5 (Oracle) results are diagnostic rather than deployable, and that Recall is lower due to the visual generation paradigm. The appendices provide substantial additional analysis — alternative pretraining city, direct visual regression comparison, matched-data-budget comparison, seed stability — that strengthens the empirical foundation beyond the main table.
Critical weaknesses
-
The coordinate recovery pipeline is insufficiently documented in the main text. The paper's central claim is about recovering trajectories, but the critical step of converting generated video frames back to GPS coordinates is barely mentioned in the body — Truth Forcing only states that the marker position is extracted. The actual geographic calibration (pixel-to-GPS transform via Web Mercator fitting) is relegated to the appendix with minimal detail. Given that this step determines the final MAE/RMSE metrics, its accuracy, failure modes, and sensitivity to map rendering parameters should be discussed in the main paper. A reader who does not consult the appendix cannot fully assess the method's validity.
-
Road-segment Recall is consistently and substantially lower than strong baselines. On Porto, DriftTraj's Mean@5 Recall (66.59%) trails RNTrajRec (67.29%) and MM-STGED (67.52%). On Chengdu, it falls below RNTrajRec by over 2 percentage points (82.02% vs. 84.17%). Even after fine-tuning, Beijing Recall (54.46%) remains below several baselines. The paper attributes this to generating trajectories over the full map image rather than performing road-segment classification, but this explanation is offered without analysis of when or why the visual approach fails to capture road-level structure. For applications where road-level accuracy matters (e.g., traffic flow estimation, turn-by-turn routing), this is a material limitation that is insufficiently addressed.
-
Beijing performance is dramatically weaker than other cities, and the cause is not analyzed. Under zero-shot transfer, Beijing's MAE (281.98m) is roughly 2.7x higher than Xi'an (94.98m) and 2.4x higher than Chengdu (115.72m). Even after fine-tuning, Beijing MAE (272.19m) remains far above the other two target cities. The paper notes that Beijing's nominal sampling interval is 60 seconds (versus 12–15 seconds for the other cities), but does not systematically analyze whether this alone explains the gap or whether Beijing's urban structure (larger road grid, different topology) poses specific challenges to the visual representation. Understanding this failure mode is important for assessing the limits of cross-city transferability.
-
The cross-city transfer claim assumes matched rendering settings, which is a strong but unvalidated constraint. The paper acknowledges in the appendix that results assume a consistent rendering protocol — including map style, zoom-level settings, and marker colors — and that invariance to different basemap providers, zoom levels, or marker colors remains unevaluated. This is a significant scope limitation: the "shared visual representation" claim is only demonstrated under a narrow rendering regime. If zoom level or map style changes between cities — which is plausible in real deployment — the transferability may not hold. At minimum, a sensitivity analysis over zoom levels or map styles would substantially strengthen the claim.
-
Unmatched model capacity and resolution. The generator has only 8M parameters operating at 96x96 pixel resolution — both are quite low. While the paper acknowledges that limited compute prevented systematic investigation of larger models and higher resolutions, the absence of any scaling analysis makes it impossible to assess whether the method's limitations (particularly Recall and Beijing performance) are fundamental to the visual approach or artifacts of insufficient capacity. A single experiment at 192x192 or with a moderately larger model would provide valuable signal.
Questions for authors
-
Beijing gap analysis. Is the large Beijing MAE primarily explained by the 60-second sampling interval (creating longer gaps between observations), or do Beijing-specific urban structure and map rendering also contribute? Have you tried subsampling Porto trajectories to a 60-second interval to isolate the effect of observation sparsity from city-specific factors?
-
Recall improvement path. Given that road-segment Recall is consistently below graph-based baselines, is there a hybrid approach that could combine the transferability advantages of visual generation with road-network constraints for segment-level accuracy? For instance, could map-matching be applied more aggressively, or could the generator be conditioned on road geometry features within the visual input?
-
Sensitivity to rendering parameters. How sensitive is transfer performance to zoom level? If the Porto-trained model is evaluated on Beijing map tiles rendered at a different zoom level than training, how much does MAE degrade? This would directly test the robustness of the "shared visual representation" claim.
-
Observation density effects. All experiments use a fixed retention ratio of 0.1. How does performance scale with observation density — for example, at 0.05 (extremely sparse) or 0.3 (moderately dense)? Does the relative advantage of the visual approach over graph-based baselines change with sparsity?
-
Candidate selection without ground truth. The Top@5 (Oracle) results show large improvements over Mean@5, but require ground truth for selection. For real deployment, is there a practical selection mechanism — for example, using trajectory likelihood under a frozen model, or consistency with observed road structure — that could approximate oracle performance without ground truth?
-
Non-vehicular trajectories. The evaluation uses exclusively taxi datasets. Has the method been tested on pedestrian or cycling trajectories, which have different movement patterns (shorter segments, more frequent direction changes, less road-network constraint)? The visual map representation might behave very differently for these modalities.
7 comments
Response to Reviewer tfFy (2/5)
Response to Weakness 3, Question 1, and Question 4:
We added statistics on the per-pixel geographic scale:
| City | Mean scale (m/pixel) | Median (m/pixel) | Min–Max (m/pixel) |
|---|---|---|---|
| Porto | 24.92 | 20.06 | 4.04–112.27 |
| Xi'an | 29.42 | 24.28 | 4.60–97.40 |
| Beijing | 29.17 | 21.73 | 5.10–200.41 |
| Chengdu | 28.59 | 21.80 | 6.07–103.02 |
The mean scales of the four cities are close, so Beijing's error cannot simply be attributed to a larger meters-per-pixel scale. We therefore tested different observation retention ratios on the four cities. The table below reports Mean@5 results. Each cell is Recall / MAE / RMSE, and the column is taken from the original paper.
| City | (original paper) | ||
|---|---|---|---|
| Porto | 60.19 / 153.41 / 194.32 | 66.59 / 110.41 / 146.54 | 79.22 / 47.99 / 72.79 |
| Xi'an | 78.86 / 151.29 / 196.95 | 85.84 / 94.98 / 130.57 | 92.36 / 34.97 / 56.00 |
| Beijing | 50.97 / 319.93 / 410.13 | 52.35 / 281.98 / 372.57 | 63.93 / 158.48 / 234.25 |
| Chengdu | 74.43 / 172.82 / 221.56 | 82.02 / 115.72 / 157.24 | 91.96 / 41.61 / 65.92 |
As observations become sparser, coordinate errors rise and Recall falls on all four cities, which shows that observation sparsity is an important factor in recovery quality. As you noted, the nominal sampling interval of the raw Beijing trajectories is 60 s. At , the gap between observations is about 600 s, and where a vehicle goes within 10 minutes is highly uncertain. Porto's nominal interval is 15 s, so reaching the same observation gap would require , at which point a large drop in performance is to be expected. We therefore believe that the sparser observations of the Beijing data are an important cause of this gap.
To directly separate the effect of observation sparsity from that of the city itself, we followed your suggestion and downsampled the Porto trajectories from the original 15 s to 60 s (matching Beijing's nominal interval), then re-evaluated with unchanged. Each cell below is Recall / MAE / RMSE.
| Setting | Mean@5 | Top@5 |
|---|---|---|
| Original 15 s, scored against 15 s ground truth (original paper) | 66.59 / 110.41 / 146.54 | 75.54 / 89.77 / 120.02 |
| Downsampled to 60 s, scored against 60 s ground truth | 47.27 / 167.34 / 224.87 | 60.72 / 124.97 / 171.73 |
| Downsampled to 60 s, scored against original 15 s ground truth | 55.07 / 198.29 / 244.49 | 69.48 / 152.07 / 192.25 |
The second row is the most comparable to the Beijing setting. After downsampling to 60 s, Porto's Mean@5 MAE rises from 110.41 m to 167.34 m and its Recall falls from 66.59% to 47.27%, even below Beijing's 52.35%. This shows that the sampling interval is one of the main factors behind the Beijing gap.
Regarding Question 4, on whether the advantage over baselines changes with observation density, we added Top@5 results under different retention ratios (each cell is Recall / MAE / RMSE, and the column is taken from the original paper):
| City | (original paper) | ||
|---|---|---|---|
| Porto | 71.44 / 121.43 / 156.66 | 75.54 / 89.77 / 120.02 | 84.66 / 41.73 / 62.92 |
| Xi'an | 87.36 / 123.49 / 162.12 | 91.64 / 80.72 / 110.62 | 96.03 / 30.07 / 47.23 |
| Beijing | 63.35 / 249.77 / 325.57 | 65.01 / 218.20 / 294.01 | 74.41 / 121.88 / 183.91 |
| Chengdu | 84.95 / 141.11 / 182.48 | 89.46 / 94.47 / 131.65 | 95.75 / 35.61 / 55.87 |
We also re-evaluated the strongest baseline, RNTrajRec, on Porto under the same retention ratios:
| Method (Porto) | ||
|---|---|---|
| RNTrajRec | 62.0 / 141.8 / 185.1 | 79.0 / 45.3 / 71.9 |
| DriftTraj Mean@5 | 60.19 / 153.41 / 194.32 | 79.22 / 47.99 / 72.79 |
| DriftTraj Top@5 (Oracle) | 71.44 / 121.43 / 156.66 | 84.66 / 41.73 / 62.92 |
At , DriftTraj Mean@5 is on par with RNTrajRec (slightly higher Recall, slightly higher MAE and RMSE). At , Mean@5 falls behind RNTrajRec. When observations are very sparse, the five candidates tend to follow different routes, and pointwise averaging pulls the result away from all of them. Top@5 outperforms RNTrajRec on all three metrics in both settings, and its margin is larger at (20.4 m lower MAE, 9.4 percentage points higher Recall) than at (3.6 m lower MAE). This indicates that the sparser the observations, the more valuable the diversity of the generated candidates becomes, and the more is lost by simple averaging. Better use of the candidates without ground truth, such as the road fusion described below, is therefore most useful in the sparse regime. These results have been added to the revision (Appendix J, Tables 16–18).
Response to Reviewer tfFy (3/5)
Response to Weakness 2, Question 2, and Question 5:
We agree that the lower road-level Recall limits the method's use in segment-level applications. The generator can currently learn the road layout from the visual input, but the training objective does not explicitly require predicted points to respect the connectivity of the directed road network. In addition, pointwise averaging over multiple candidates can pull the output to somewhere between two candidate roads. We believe these are the main reasons for the lower Recall, and the revision now states them in Section 5.2.
To explore combining visual generation with road constraints, and to address your Question 5 on how to make use of multiple candidates without ground truth, we added a road-fusion method that requires no ground truth for the missing positions. It keeps the five candidates generated by the frozen model and builds an additional reference route from the known anchors and an external directed road network.
Specifically, longitude and latitude are first projected onto each city's local UTM metric plane. Candidate road positions are identified near adjacent known anchors, and a connecting route is searched on the directed road network. The cost of road edge is
\qquad \rho=0.5,\quad v_0=50,$$ where $L_e$ is the edge length and $v_e$ is a fixed priority value assigned by road class. It expresses a preference over road classes and is not a measured travel speed. The search also takes into account the distance from each anchor to its candidate road positions. (We use $\rho$ for the road-weight exponent to avoid confusion with the observation retention ratio $\gamma$.) Once the route is determined, we interpolate along its arc length according to the relative time between adjacent observations, obtaining $\mathbf p_{\mathrm{road}}(t)$. The five model trajectories are projected onto the same metric plane and averaged to give $\bar{\mathbf p}_{\mathrm{model}}(t)$, and we then compute $$\mathbf p_{\mathrm{fused}}(t) =w\mathbf p_{\mathrm{road}}(t) +(1-w)\bar{\mathbf p}_{\mathrm{model}}(t), \qquad w=0.75.$$ Finally, the known observed positions are restored and the result is projected back to longitude and latitude. $\rho$ and $w$ are selected on the Porto validation set and are not tuned again when transferring to other cities, so all cities share the same values. If the road search cannot connect two anchors, we fall back to a direct connection. The road-fusion results are shown below. Each cell is Recall / MAE / RMSE. "Best baseline" takes, for each metric, the best value among all baselines in Table 1 of the revision, and the last column is the relative gain of the fused result over it (increase in Recall, reduction in MAE and RMSE). | City | Best baseline | DriftTraj Mean@5 | DriftTraj + road fusion | Improvement over best baseline (%) | |---|---:|---:|---:|---:| | Porto | 69.84 / 102.6 / 140.6 | 66.59 / 110.41 / 146.54 | 74.81 / 102.11 / 137.59 | 7.1 / 0.5 / 2.1 | | Xi'an | 85.91 / 93.1 / 126.5 | 85.84 / 94.98 / 130.57 | 90.96 / 75.11 / 103.45 | 5.9 / 19.3 / 18.2 | | Beijing | 56.61 / 278.6 / 373.7 | 52.35 / 281.98 / 372.57 | 60.99 / 230.77 / 306.78 | 7.7 / 17.2 / 17.9 | | Chengdu | 84.17 / 95.1 / 131.8 | 82.02 / 115.72 / 157.24 | 90.59 / 87.49 / 119.45 | 7.6 / 8.0 / 9.4 | After fusion, Recall, MAE, and RMSE improve over Mean@5 on all four cities, and the fused result is better than the best baseline value on every metric in every city. Relative to the best baseline, Recall increases by 5.9–7.7%, MAE decreases by 0.5–19.3%, and RMSE decreases by 2.1–18.2%. This shows that in applications where a road network is available, a road prior can further improve the output of the visual model, and the whole procedure requires no ground truth. It is reported in the revision as an additional hybrid variant (Appendix K, Table 19), clearly separated from the purely visual method.
Response to Reviewer tfFy (1/5)
We thank Reviewer tfFy for the careful review. We respond to each point below. All changes in the revised PDF are marked in blue.
Response to Weakness 1:
We agree that this step directly affects the final metrics and should be explained in the main text. We have added it to the main text of the revision (end of Section 4.3), with the full formulas in Appendix A.5. The recovery pipeline consists of, in order: marker decoding, temporal alignment, and a deterministic pixel-to-geographic transform.
First, the marker's pixel position is extracted from each generated frame using the red response and weighted centroid defined in the paper. If extraction fails, the previous position is kept. Temporal resampling and coordinate alignment are then performed under the observation constraints, yielding a predicted pixel position for each query time. The geographic transform is calibrated from the pixel/GPS pairs of the known observations. The fit does not use the ground-truth coordinates of missing positions.
Let S be the side length of the original square image and r=96 the side length of the network input. A known pixel point (ui,vi) is first mapped to the network pixel scale with the vertical axis flipped:
xi=(Srui,−Srvi).
The corresponding known longitude and latitude (λi,φi) are projected to Web Mercator, with angles in radians and R=6,378,137 m:
\left( R\lambda_i, R\log\tan\left(\frac{\pi}{4}+\frac{\varphi_i}{2}\right) \right).$$ Because the maps are north-up square crops, we fit only a shared scale $s$ and a 2D translation $\mathbf b$, rather than an arbitrary affine transform: $$(s,\mathbf b)=\arg\min_{s > 0, \mathbf b} \sum_i\left\Vert\mathbf m_i-(s\mathbf x_i+\mathbf b)\right\Vert_2^2.$$ The closed-form solution is $$s= \frac{\sum_i(\mathbf x_i-\bar{\mathbf x})^\top(\mathbf m_i-\bar{\mathbf m})} {\sum_i\Vert\mathbf x_i-\bar{\mathbf x}\Vert_2^2}, \qquad \mathbf b=\bar{\mathbf m}-s\bar{\mathbf x}.$$ For a predicted pixel $(\hat u_t,\hat v_t)$ at the network scale, we first compute $$(\hat X_t,\hat Y_t)=s(\hat u_t,-\hat v_t)+\mathbf b,$$ and then recover longitude and latitude through the inverse projection: $$\hat\lambda_t=\frac{\hat X_t}{R}, \qquad \hat\varphi_t=2\arctan\left(\exp\left(\frac{\hat Y_t}{R}\right)\right)-\frac{\pi}{2}.$$ Finally, radians are converted to degrees.