- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the conversion of piping and instrumentation diagrams (P&IDs) into structured listings containing identifiers, connectivity, and engineering attributes. PID-AGENTS first uses OCR and symbol detection to locate pipeline identifiers and instrument tags, and retrieves project specification documents to prefill engineering attributes. It then processes individual pipelines or instruments using Gemini 3 Flash with tools for local image cropping, text search, line-segment localization, event-driven tracing, and connectivity probing to recover endpoints and associated attributes. The paper reports a cross-project evaluation from Project A to Project B, with a quantitatively annotated subset containing 212 pipeline connections and 622 instrument instances. The main contribution is a tool-augmented parsing workflow for real industrial drawings, together with empirical validation in this application.
Critical strengths
- The problem has practical value, and the task goes beyond symbol detection. The paper integrates pipeline endpoints, instrument-to-pipeline associations, and specification attributes into a unified output (Sections 2–3), which more directly addresses the needs of engineering listing workflows. Section 3.3 describes a reasonable division of labor: tools trace line segments, while semantic decisions at junction or endpoint events are informed by local observations. Two execution traces on OPEN100 drawings in Appendix D illustrate this workflow.
- Results on real industrial drawings provide positive evidence for the complete workflow. Table 1 includes component-level comparisons involving perception, RAG, and the full agent stage. For pipeline listing, specification retrieval primarily improves specification fields, while adding the agent and drawing tools increases From/To accuracy from 45.75%/41.51% with VLM + Perception + RAG to 63.21%/66.51%. The average field accuracy for instrument listing also increases from 48.96% for the best baseline reported in the table to 77.55%. These results support the overall value of the tool-augmented stage, although they do not isolate the contribution of each tool.
Critical weaknesses
- The incremental benefit of the specialized topology tools over general local inspection remains unclear. Section 3.3 describes the interaction between event-driven tracing and agent-based endpoint identification, and Table 1 supports the gains from the complete tool-augmented stage. However, the reported experiments do not separately compare an agent using only local crops and text search with the full agent that additionally uses line-segment localization, tracing, and connectivity probing. Consequently, it remains unclear how much of the improvement comes from clearer local visual inputs and additional inference computation, and how much comes from the topology tools emphasized in the paper. The implementation descriptions in Section 3.3 and Appendix B.1 also remain relatively high-level. For example, rules for handling line crossings and breaks, and criteria for filtering candidate contact points, are not specified in detail, limiting assessment of the tool designs and their operating conditions.
- The end-to-end evaluation of connectivity requires further specification. Section 4.1 (L365–371) explicitly evaluates From and To separately and uses partial matching for endpoints, but neither the main paper nor the appendix details the matching rules or clearly explains how missed identifiers, extra predicted records, and timed-out records are counted. The 88.57% in Table 1 is the mean accuracy across pipeline fields, including several specification fields with near-perfect scores; the From/To values of 63.21%/66.51% are accuracies for the two endpoint fields. These metrics do not directly provide the proportion of pipeline connections for which both endpoints are correct. Clarifying endpoint normalization and semantic-equivalence rules, reporting the accuracy of complete ordered endpoint pairs using existing predictions, and explaining the treatment of incomplete records would help assess the reliability of the resulting connectivity listings.
Questions for authors
- Could the authors compare an agent using only local crops and text search with the full agent that additionally uses line-segment localization, tracing, and connectivity probing, while keeping the OCR/detection outputs and VLM fixed and using comparable model-call budgets? Reporting From/To accuracy and the proportion of connections with both endpoints correct under clearly defined matching rules would help establish the incremental contribution of the specialized topology tools over general local inspection.
2 comments
Response to Reviewer LxwS (2/2)
End-to-End Evaluation of Connectivity (W2, Q1.2)
R2 Using the existing predictions, the table below reports the accuracy of complete ordered endpoint pairs (Both), together with From and To, under strict matching and under the partial matching of Table 1 (new Appendix C, Table 8).
| Method | From (strict) | To (strict) | Both (strict) | From (partial) | To (partial) | Both (partial) |
|---|---|---|---|---|---|---|
| VLM baseline | 0.3019 | 0.1840 | 0.0660 | 0.4623 | 0.4009 | 0.2642 |
| VLM + Perception | 0.2925 | 0.2123 | 0.0896 | 0.4387 | 0.4104 | 0.2453 |
| VLM + Perception + RAG | 0.3066 | 0.2123 | 0.1179 | 0.4575 | 0.4151 | 0.2547 |
| Rule-based Controller | 0.2075 | 0.3066 | 0.0849 | 0.3019 | 0.4104 | 0.1509 |
| Pixel-Graph Search | 0.2594 | 0.2830 | 0.0943 | 0.4906 | 0.4340 | 0.2311 |
| Rule-based Topology | 0.1934 | 0.2264 | 0.0377 | 0.3113 | 0.3349 | 0.1226 |
| PID-Agents | 0.4387 | 0.4481 | 0.2075 | 0.6321 | 0.6651 | 0.4953 |
PID-Agents gets both endpoints right for 49.5% of the pipelines under partial matching and for 20.8% under strict matching. No other method exceeds 26.4% and 11.8%, respectively.
Endpoint normalization and semantic-equivalence rules. The ground truth for From and To comes from the project line list, which is compiled by hand. The list often names an endpoint differently from the label on the drawing. For example, it records the group name "P-101A/B" for a line that starts at pump P-101B alone. Both strings are first uppercased, and runs of whitespace are collapsed. A predicted endpoint is then correct if it equals the ground truth, if one string contains the other, if the two share an equipment tag after a group such as "P-101A/B" is expanded, or if both contain the header token "HDR". Strict matching instead requires the first predicted endpoint to equal the ground truth exactly after uppercasing and trimming spaces. Both counts ordered pairs. A pipeline counts only when the predicted From matches the ground-truth From and the predicted To matches the ground-truth To, so a swapped pair counts as wrong.
Incomplete records. The new Appendix C states three rules. A missed identifier, which is a ground-truth pipeline that a method does not list, counts as wrong in every field. An extra predicted record, which has no ground-truth row on its sheet, does not affect accuracy. A timed-out record is scored as wrong. In the PID-Agents runs of Table 1, every listed pipeline returned an answer. PID-Agents lists 209 of the 212 pipelines, and each of the three missing ones continues across two adjacent sheets.
Mean accuracy across pipeline fields. We agree that the 88.57% mixes near-perfect specification fields with the two endpoint fields. We therefore report From, To, and Both separately in the new Table 3 and Table 8, and our claims about connectivity now rest on these columns.
We hope that these results address your concerns, and we would be grateful if you would consider raising your score.
Response to Reviewer LxwS (1/2)
We thank the reviewer for realizing the practical value of the task and its unified output, the reasonable division of labor between the tools and the semantic decisions, and the positive evidence on real industrial drawings. We address each concern below with new experiments and revisions, which are highlighted in blue in the revised paper.
R1 We ran this comparison on a subset of the evaluation set with 64 pipelines and 193 instruments (revised
Section 4.4, newTable 4). The OCR/detection outputs and the VLM are fixed, and only the tool list changes. All scores use the partial matching ofTable 1, whichR2defines.LLM calls is the mean number of model calls per pipeline on this subset. The model-call budgets are comparable. Adding line-segment localization and tracing raises Both from 43.8% to 59.4% with only 2.3 more model calls per pipeline. The full agent reaches 56.3% with 4.5 more calls. The difference between the last two rows is two of the 64 pipelines. The agent calls Connectivity Probing only about 0.2 times per pipeline, and repeated runs show that this difference is within run-to-run variation (new
Appendix B.4).Additional inference computation does not explain the gain. Configurations without ROI Rendering use 37 to 85 model calls per pipeline, yet none of them exceeds 45.3% Both (
Table 4). Local crops and text search alone give 43.8% Both, and the specialized topology tools add 12.5 to 15.6 points at a comparable model-call budget.Rules for line crossings, breaks, and candidate contact points. The new
Appendix B.4specifies these rules.