Reality Reskin

Overview

Reality Reskin explores real-time video restyling for AR using Decart's LSD model. The goal: make passthrough on Quest look like you're peering into another dimension with perspective-correct rendering.

I knew creating a true reality reskin was impossible in a weekend, but seeing is believing. If I could get close, I could figure out what it takes to go from prototype to the ideal version: hyper-real, 90fps, persistent reality reskinning. Read the full writeup

Successes

Solved for perspective-correct rendering by approximating an eye with a slice of a sphere, then warping the output until the restyled feed aligned with real-world vision. This trick enabled the ELSEWHERE experience that won 3rd place in the Decart hackathon.

Shipped an experimental .apk. Current limitations include low persistence (objects fading in/out), single-eye input, and high render latency (11-38ms). But after experiencing it, I'm confident this sort of real-time restyling will be viable in the next few years.