Magic Windows - Live VR Body Painting

Exploring the boundaries of augmented and virtual reality with live body painting projection.

Recently I’ve been exploring drawings softwares like Tilte Brush and Quill, that allow to use VR as a creative medium while inside VR.

For this projection mapping/augmented reality assignment, I decided to do a live VR painting that would be projected into my body.

vr body mapping.JPG

The projector setup was very basic, with it standing on a tripod facing me. For the time I didn’t use any live mapping software, since the challenge was how to make me - the artist - aware of my surroundings and myself being projected upon while immersed inside the VR painting program.

For this I’ve found an app called OVRDrop. Together with Virtual Desktop, these software re-creates your desktop inside any VR environment.

ego vr setup.JPG

At first I used a webcam and later a DSLR camera connected to my computer to feed OVRDrop with external footage of myself. This allowed me to carefully move my head and arms in order to properly align my drawings with the projection.

Projector connected to youtube stream. Webcam taped on top of it - later replaced by DSLR camera.

Projector connected to youtube stream. Webcam taped on top of it - later replaced by DSLR camera.

I also experimented with using a YouTube live streaming to connect the projector with what I was seeing/drawing inside the headset. I was also able to see the streamed video inside the VR painting app. This created some very interesting, albeit disconcerting experiences as I tried to synchronize my movements with the delay of the streamed video. One possibility is to further extrapolate this multi body / multi temporal perspective, allowing for several streams and projectors to be connected with my VR point of view.

Having someone to help setup and position myself was essential. Thanks Dan!

Having someone to help setup and position myself was essential. Thanks Dan!

As of this being VR or AR, the whole goal was to extrapolate the gradient between both. Artist and viewers are both seeing the same animation, but they are in different positions of the AR/VR gradient. I’m immersed in VR while I see and manipulate my “augmented” self.

0C4A7220.JPG

Next Steps

For future installments, I’d like to allow the viewers to interact further with the experience, either on location or through streaming services.

Use computer vision to track my body and features, allowing me to move and interact with the live VR drawings being created.