Cyrus Vachha will present his General Exam "Enabling Transitions Across Physical Reality and Virtual Representations of Spaces" on Friday, May 1, 2026 at 10:00 AM in CS 301 and via zoom.

Zoom link: https://princeton.zoom.us/j/98265539965?pwd=SNOZdUkcuap7MnyDv6zS2ahotL8KFc.1

Committee Members: Parastoo Abtahi (advisor), Felix Heide, Szymon Rusinkiewicz

Abstract:

Recent advances in Gaussian splatting, 3D foundation models, and video-passthrough AR/VR offer unprecedented fidelity for capturing and blending physical and virtual worlds. Yet, a fundamental limitation remains: the inability to act upon these layered representations of reality with physical reality itself. While we can visualize environments with photorealistic detail, current systems lack the architectural support to make these captures truly interoperable with agents or physical actions. To bridge this disconnect, I reconceptualize XR as a multi-layered reality, where the challenge lies in facilitating bidirectional transitions: allowing interactions to propagate from physical reality to virtual representations and back again.

I first present SplatXR, a Unity toolkit bridging the gap from physical reality to visual virtuality. By providing building-block abstractions, SplatXR enables developers to build end-user applications leveraging splats and facilitate transitions across layers, such as reconstructing real objects into splats or converting static captures into interactive mesh-enhanced splats. This effectively transforms passive captures into a programmable reality. I discuss future research directions extending this framework toward controllable, situated world models for composing generative interactable elements implicitly in XR.

Building on this layer traversal framework, I present preliminary concepts for a robotic tele-operation system demonstrating the inverse transition: where virtual interactions propagate back into physical reality. By selectively rendering 3D portals of remote scenes and increasing visual and instruction fidelity where robotic uncertainty is highest, the system would allow users to resolve linguistic ambiguities through spatial interaction. This bidirectional interoperability creates a natural interface for communicating with reality, illustrating a future where the digital-physical boundary becomes a traversable interface.

Reading List:

https://docs.google.com/document/d/1XrDTOyRBRv-oP8zvbjmT-efdVy6iBvW2xWRoAnKxBrs/edit?usp=sharing

Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.