- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors propose Elevator-VIGS, a VI-based 3DGS SLAM system for use through elevator rides. Unlike earlier works which use LiDAR or other sensors, they do so using vision and IMU. The core challenge here is that the RGB and IMU data are conflicting inside the elevator, making pose estimation challenging. They address this by using a transport state which separates elevator and robot motion, detecting elevator rides using zero-shot VLM detection and anchored by arrival and departure (0 velocity) conditions of the elevator. The system is evaluated on a wide range of both sim and real scenes, both enclosed and glass elevators, and substantially outperforms prior work, on the evaluated sequences, while generating a GS map.
Critical strengths
- The paper addresses a challenging problem of SLAM in elevators, and to my knowledge is the first GS SLAM system to do so.
- The paper is comprehensive and well evidenced, providing real world experiments in a variety of elevators, including glass ones.
- The system performs well on reported sequences and is compared against a wide range of methods using different modalities.
Critical weaknesses
NIL.
Questions for authors
- How does the system perform in glass enclosed or semi-glass elevators? The authors do report numbers for the detector, P_{elev} drops 0.91->0.82 from enclosed to glass, while only triggering on 75% of frames in the glass ones, vs every frame for enclosed. But how does the transport state handle this? In Sect. 3.2.1, it says the visual residual is written onto the elevator-frame and not discarded, so it seems like it would conflict with the inertial residuals. Yet the results suggest that system works fine in practice, so I would like to see some numbers on how the transport state performs in glass elevators, and how it handles the visual residuals in this case.
- C.2, C.4 seem to imply the system can handle elevators which stop at multiple floors, but I don't see any sequences that test this? And how does it perform (i.e. Eq. 14) when the camera starts moving before the elevator stops at a floor, which is likely human behavior to pre-empt getting out.
- The only mention of people in this text is the blurring of faces. Did any of the real world sequences include people in the elevator? I am curious how the system handles people or other objects in the elevator, which is the most common case in the real world? Presumably, this affects both elevator detection of the VLM and visual tracking in elevator frame.
3 comments
Response to Reviewer YL9F (2/2): several stops, moving before the elevator stops, and people in the elevator
Q2. Several stops, and moving before the elevator stops. Both cases work with the same code and configuration as every other sequence, and every ride is detected. The paper indeed has no sequence with an intermediate stop, since its two-ride sequences are round trips, in which the robot rides to another floor, leaves the elevator and later rides back to the starting floor. We therefore recorded new sequences.
Several stops. Our formulation supports intermediate stops without any modification. Each arrival ends a ride and each departure begins a new one (App. C.4), so a trip with a stop is detected as two rides.
In two new real-world sequences, the elevator waits 11 s and 26 s at an intermediate floor with open doors and then continues. The ride detector finds both rides of each sequence and reports no false ride during the wait. It detects a departure from the IMU only when the vertical velocity stays above 0.4 m/s for 1 s, which does not happen while the elevator stands at a floor, with open doors or with passengers moving. The vertical error over the whole trip stays within 2.1% of .
- New office building, = 14.28 m. The error is 7 cm with the stop (27 cm for a non-stop ride in the same elevator).
- Residential2's elevator, = 49.97 m. The error is 102 cm with the stop (62 cm for a non-stop ride).
Moving before the stop. Our formulation also supports this case without any modification, and it hurts neither detection nor accuracy.
The robot's motion and the elevator's motion are tracked separately. The transport state carries only the elevator's vertical motion. The camera and the IMU together track the robot relative to the elevator, in rotation, horizontally and vertically. A robot that walks to the doors while the elevator decelerates is therefore tracked like any other motion inside the elevator.
Three new real-world sequences test this case. In the 4 s before the stop, the robot moves 21 to 88 cm and turns 51° to 95° (at most 16 cm and 25° in the paper's sequences). The arrival is detected in every ride. The one-way vertical errors, measured against the laser-measured floor height , are 0.5%, 1.9% and 2.1% of , within the 2.9% of the paper's one-way sequences.
Eq. 14. Eq. 14 gives the error when the robot itself moves vertically during a rest of the elevator. Our code now determines the rise from two rest windows, the few seconds in which the elevator stands still before the departure and after the stop (see our general response). The robot's velocity at the instant of the stop therefore no longer enters the rise. Only the robot's mean vertical velocity over a whole rest window still does, and it changes the rise by , where is the duration of the ride.
Q3. People in the elevator. Our system handles passengers in the elevator, and every ride with passengers is detected and tracked within the accuracy of the paper's sequences. One of the paper's sequences, Station3, has other passengers in the elevator, and we recorded three more sequences in which passengers stand and walk in front of the camera. One of the three has a mirror that reflects the passengers.
Detection. Passengers do not affect the detection of the VLM, and all five new rides are detected with no false ride reported. The median is 0.78 to 0.95 and stays above the 0.5 threshold on every frame of the rides. The departure and the arrival are then detected from the IMU, which passengers do not affect.
Tracking. Passengers who stand still move with the elevator, so the visual residual treats them like the rest of the elevator. Walking passengers get no dedicated term. The BA weighs each visual residual by its confidence, and the pixels on walking passengers receive 3 to 5 times less confidence than the rest of the image. The inertial residuals keep constraining the robot's rotation and horizontal motion. The one-way vertical errors of the new sequences lie within the 2.9% of the paper's one-way sequences.
I appreciate the authors’ detailed responses. The additional experiments address my earlier concerns. Overall, I find the real-world value of LiDAR-free operation compelling, but certain ideas are borrowed from Elevator-LIO and applied to visual-inertial sensing. As such, I will retain my score.
Response to Reviewer YL9F (1/2): glass elevators
Q1. Glass elevators. Glass elevators work as well as enclosed ones in tracking and in detection, and the visual and inertial residuals do not conflict. We answer the three parts of the question in turn.
Performance. The table compares the vertical error of the two elevator types on the paper's 16 real-world sequences.
On a one-way sequence, the robot rides to another floor, and the vertical error is the difference between the estimated height change and the laser-measured floor height H. On a round-trip sequence, the robot rides away and returns to the starting floor, and the vertical error is the estimated height difference on returning to the start. The range is taken over the sequences of each row.
On the one-way sequences, the two glass rides have errors of −1.7 cm on Office2 (0.1% of H) and −7.0 cm on Station1 (1.0%), below the enclosed mean of 22 cm. On the round trips, the three glass sequences Office3, Station2 and Station3 return to the start with errors of 0.4, 10.2 and 15.6 cm. Their largest error is about the largest enclosed one (16.0 cm), and the two means differ by 5 cm.
Detection on 75% of the frames. This does not hurt detection, and all five glass rides are detected. The two visual cues are not needed on every frame. They only have to fire together for 1 s to tell the detector that the robot is inside an elevator, and the departure and the arrival are then detected from the IMU. The frames on which the cues do not fire therefore cost nothing.
Visual and inertial residuals. In practice the visual residuals do not conflict with the inertial residuals, because the pixels that look through the glass receive low confidence. As in DROID-SLAM, the BA weighs each visual residual by its confidence. Once the elevator departs, the confidence of the pixels that see through the glass drops by 80 to 88%. Their visual residuals are kept, as Sec. 3.2.1 says, but they carry little weight. The pixels on the elevator itself dominate, and these pixels agree with the elevator frame.
The elevator's rise that the transport state holds comes from the IMU, so the view through the glass does not enter it.