XRStudio (CHI 2021)

XRStudio enables an instructor to set up a mixed reality capture area for streaming their video feed superimposed with virtual 3D content to students, allowing instructors to give lectures in VR and students to access the immersive content with or without VR devices.

XRStudio: A Virtual Production and Live Streaming System for Immersive Instructional Experiences

There is increased interest in using virtual reality in education, but it often remains an isolated experience that is difficult to integrate into current instructional experiences. In this work, we adapt virtual production techniques from filmmaking to enable mixed reality capture of instructors so that they appear to be standing directly in the virtual scene. We also capitalize on the growing popularity of live streaming software for video conferencing and live production. With XRStudio, we develop a pipeline for giving lectures in VR, enabling live compositing using a variety of presets and real-time output to traditional video and more immersive formats. We present interviews with media designers experienced in film and MOOC production that informed our design. Through walkthrough demonstrations of XRStudio with instructors experienced with VR, we learn how it could be used in a variety of domains. In end-to-end evaluations with students, we analyze and compare differences of traditional video vs. more immersive lectures with XRStudio.

Acknowledgements

This work was supported in part by a research grant from the Center for Academic Innovation as part of the XR Initiative. Research related to this project was also partially supported by a Mega Grant from Epic Games. We are very grateful!

Full Citation (student names are underlined)

XRStudio: A Virtual Production and Live Streaming System for Immersive Instructional Experiences (CHI 2021)
M. Nebeling, S. Rajaram, L. Wu, Y. Cheng, J. Herskovitz

Accompanying Video

Next Steps