We are happy to announce our CHI 2021 paper on XRStudio, a virtual production and live streaming system allowing instructors to give lectures in VR, and enabling students to access the VR content with or without VR devices. This project involved several students from the lab: Shwetha Rajaram, Liwei Wu, and Jaylin Herskovitz, and a visiting student from Swarthmore College, Yifei Cheng. They co-presented the paper in this CHI video presentation. Michael and Shwetha will be at CHI to answer questions during the two presentation slots.
Together with Professor Mark Billinghurst, Michael also developed and co-taught a CHI 2021 course on XR prototyping. In preparation for the course, Mark and Michael have started to collect more resources for those interested in rapid prototyping techniques for XR, which can be found at http://xr-prototyping.org/.
We are looking forward to catching up with friends at CHI 2021 which will be held virtually this year from May 8 to May 13.
The postdoc should be experienced in technical human-computer interaction research, ideally demonstrated by a set of significant publications in the ACM CHI, UIST, or EICS communities. Notably, you don’t have to be experienced in AR/VR and you don’t need to have a Computer Science background to work with Michael and the lab.
Michael’s web site and our lab web site share a lot of the work we have worked on and we are always excited to talk about some of the work we are doing at the moment as well.
CIFellows 2021 awardees will receive a two year postdoctoral opportunity in computing, with cohort activities to support career development and community building. Realizing there are still many unknowns with the pandemic and that situations are different across the nation, there will be some flexibility in the program.
Applications are due around early May 2021, with decisions being made by July 1, 2021 for positions beginning this fall or next winter. Please check the website (https://cifellows2021.org) regularly for application requirements and submission details.
There are three courses in the Extended Reality for Everybody specialization on the knowing, doing, and shaping of XR. The first course provides an introduction to the fundamental concepts and technologies and a discussion of key issues and trends. The second course covers a variety of prototyping and evaluation techniques for user experience and interaction design. The third course focuses on the development of XR applications.
Note: U-M faculty, staff, students, and alumni can access the specialization through Michigan Online for free.
Professor Michael Nebeling and the Information Interaction Lab are happy to announce that the Extended Reality for Everybody specialization, or XR MOOC series as Michael likes to call it, has launched this week on Coursera!
UPDATE: The XR MOOC can now also be accessed through Michigan Online (free access for U-M faculty, staff, students, and alumni).
This is a set of three courses that together provide an overview of the XR technology landscape and a discussion of the key issues and trends (Course 1), take a design thinking and doing approach to creating new XR experiences more suited to designers (Course 2), and also cover three main approaches to developing XR applications with WebXR using A-Frame, Unity, and Unreal (Course 3).
The design and production of this XR MOOC series was a collaborative, 1-year effort, that was impacted by COVID-19 in that Michael had to film the majority of the MOOC by himself and in his study. Michael thinks of this as an advantage, however, because it adds a personal touch like featuring his pet rabbit, his companion during filming and the main reason for the many interruptions, in the Making Of at the end of Course 3.
We hope you will take advantage of these online teaching resources and, if you are a researcher and instructor, also share it with your students! Please let Michael know if there are any good examples and resources that you think could be added to what is currently there.
The Information Interaction Lab at the University of Michigan School of Information is looking for one master student to assist on an ongoing VR project led by PhD student Audrey Labrie, working in collaboration with Professor Nebeling. This position will be a paid temporary research assistant position where the amount of work can typically vary but the expectation will be no more than 12 hours per week.
We are looking for a student with prior industry or research experience conducting qualitative user research. The student will engage in fundamental research tasks to help address the goal of the project. In particular, the student will assist in the execution of a human-subjects study, qualitative analysis of data from the study, and possibly contribute to the formal write-up of a publication about the research.
Required qualifications:
Prior industry or research experience conducting qualitative user research
Knowledge of qualitative research methods
Autonomous and reliable
Experience with AR/VR technologies (e.g., through courses like SI 559 / 659) is a plus but not required. If you think you would be a good fit for this project, please send an email to Audrey Labrie (alabrie@umich.edu). Be sure to include a short paragraph about yourself and describe your involvement in a qualitative user research project. Please also include your CV. Interviews can start in December and decisions will be finalized in January.
We are delighted to support Jeremy Nelson, Director of the U-M XR Initiative, in kicking off his MiXR podcast. In this first episode, he sat down with me, Michael Nebeling. We discuss a lot of the developments around XR, both at U-M and outside. Some of the key points included how I found my own way into XR, and into teaching it, and I also give examples from recent research and share my perspective on the future. I really enjoyed the conversation and I hope you do, too.
Professor Nebeling and the Michigan Information Interaction Lab are thrilled to announce that Audrey Labrie has decided to join the UMSI PhD program. Audrey will be joining us from Polytechnique Montreal where she is currently finishing up her undergraduate in Software Engineering.
Audrey has a variety of interests in mixed reality interfaces, interaction techniques, accessibility, and how to learn and be productive with these evolving new sets of technologies. Recently, Audrey has been visiting with Professor Albrecht Schmidt‘s research group at LMU in Germany, where she helped create and study an mobile augmented reality German language learning app with users. The project was supervised by Lewis Chuang, Fiona Draxler and Albrecht Schmidt, and led to a publication at CHI 2020.
Audrey is currently working with Professor Jinghui Cheng at the User-Centered Design Lab at Polytechnique Montreal on two user-centered AR design projects.
We are looking forward to welcoming Audrey to U-M, the UMSI community, and our lab, and to many exciting new research projects together with her!
Professor Nebeling and the Michigan Information Interaction Lab are thrilled to announce that current MSI student, Shwetha Rajaram, has decided to join the UMSI PhD program.
Update: 2/3 positions have been filled. But we have one more opening for summer 2020! For this last position, we’re really looking for someone technical.
Summary: The Information Interaction Lab is offering up to 3 summer internship positions to students who would like to contribute to our ongoing XR design projects, some of which will be targeted for publication at the CHI 2021 conference. We also respond to a call from UMSI-CDO and highly encourage UMSI students who are experiencing distress and uncertainty about their planned summer internships due to COVID-19. We are committed to providing the following:
Opportunities for applied research methodology and skill-building; and/or work that can support the research team in some way (students will assist with the development of research prototypes in the lab; Professor Nebeling will provide a framework for the internship and guide students through each milestone)
Supervision that includes mentorship by someone that is knowledgable of the intern’s work and can provide adequate guidance, support, and learning (Professor Nebeling works with all students directly, with weekly project and mentoring meetings)
Access to external constituents, partners, and/or users (unfortunately, this will be limited given the COVID-19 pandemic but we will be conducting research that is aligned with previous research gifts from Mozilla and Disney Studios)
Learning that will benefit the intern in developing their skillset and professional experience, as well as support for meeting their learning objectives (we will have weekly virtual lab meetings to talk about project goals, ideas, and milestones)
Fair compensation: The average on-campus internship pay rate for master’s students is $15-$20/hr (we will commit to a pay rate of $20/hr and will have work of 15-20 hours per week, on average, but actual workload may vary and the expectations will be commensurate with each individual student’s personal & work situation)
More details about the projects below:
Working with UMSI professor Michael Nebeling in the Information Interaction Lab, the projects we are currently working on, at the highest level, are investigating how virtual and augmented reality interfaces can be created more easily and faster, and how XR (where X is a placeholder for VR and AR) can become more effective modes of interaction and complement existing interfaces. Our current focus as part of our involvement with the U-M wide XR Initiative is on use cases that support (remote) immersive learning and instruction through the use of XR technologies. Depending on the student’s experience and interests, they could help with the development of new tools to support this investigation, the design and evaluation of new kinds of AR/VR interfaces with a particular focus on novel interaction techniques, or both.
Some of the new research questions Professor Nebeling is asking include: How do we abstract from the complexities of tools like Unity and Unreal, yet allow designers over a short period of time in some of our tools to transition to these more advanced tools? How can we prototype complex, interactive AR/VR experiences without writing code, mostly using programming-by-demonstration? How can we enable artists to fully express their designs on paper, or in immersive authoring tools using either AR, or VR, or some combination of both? How can we appropriate complex physical objects and use them to define complex interactive behaviors in AR or VR while occluding and potentially deforming them in the process? What would a WordPress for AR/VR look like in terms of enabling concepts and the most useful getting started templates? How can we adapt a tool like PowerPoint, which works in 2D, to create “immersive slideshows” for AR/VR, without the need to design in 3D?
Desired experience & skills
The focus of the project will be determined based on student interest and experience, and can range from more technical development to more user research oriented angles. Experience shows that students with a technical background such as Computer Science do very well when working with Professor Nebeling and other lab members. Ideally, you have…
successfully completed Professor Nebeling’s AR/VR courses, SI 559 and SI 659
previous programming experience–either acquired through coursework or, ideally, through projects–with web technologies such as HTML5/CSS3, JavaScript/jQuery, and Node.js
familiarity with THREE.js/A-Frame and JavaScript, Unity and C#, and/or Unreal and C++ with Blueprint
experience with and access to* AR/VR platforms and devices (ARKit/ARCore, Oculus Rift/Quest, HTC Vive, Microsoft HoloLens, etc.)
* We will consider ways of allocating and distributing AR/VR equipment so that students can work on projects remotely.
Roland Graf, associate professor at the University of Michigan Stamps School of Art & Design, and Michael Nebeling, assistant professor at the University of Michigan School of Information, with Bryan Kreps during a “play test” for iGym. (Photo by Roger Hart, Michigan Photography)
Professor Michael Nebeling was part of the U-M team that created a first version of iGYM, a system designed for school and community-based sport or recreation facilities seeking to provide novel and accessible ways for people with different levels of mobility to play and exercise together. In our current implementation, iGYM produces an interactive gaming floor for inclusive play of a “life-size version of air hockey” using a simple form of projected augmented reality.
Funded by U-M’s ESSI program, the project achieved many of the goals that are envisioned by U-M’s exercise and sport science initiative. It involved the target audience and demonstrated large impact on the users that helped us test and improve it, in this case, the kids, their friends, and their parents. The project also received a best paper award at CHI Play 2019, and wide media coverage in December 2019 (e.g., in U.S. news, ABC news, VRScout, as well as in local news papers).
Most media articles provide Professor Roland Graf‘s perspective, our friend and collaborator and the lead researcher on the project. For Professor Nebeling, it was a very rewarding project to be a part of and something he won’t forget. While it is of course nice to have helped the team win a best paper in the HCI research discipline, the professional and personal impact on Professor Nebeling was much larger than that.
First, he was able to recruit a first selection of UMSI master’s students taking his brand new AR/VR courses, SI 559 and SI 659, Pallavi Benawri and Amy Whitesall, as research assistants. Both really played a significant role in the success of this project!
Second, with most research projects, a researcher is happy when a user study produces great results and demonstrates effectiveness of the system design. With iGYM being a system that is quite complex and difficult to study given the various design parameters, it was a little different. While overall received positively, our user study revealed some mixed feelings about the system. Despite the use of algorithms and manual fine-tuning to help adapt the difficulty and balancing model of the system, it remains a challenge to make it both fun and fair, especially since with the range of parameters we can control, it was quite easy to “overbalance” the system. The results helped us understand how we could make the system better in the future, and this is what we consider a success.
But perhaps the greatest success of this work was that we could begin to see how iGYM might be adopted in the future. To this end, Professor Nebeling really enjoyed helping organize and host two play days with the larger team, including our consultant Betsy Howell, the kids, and of course their parents. It was an amazing atmosphere and really interesting to see our research prototype in action, having our participants “break test” it through creative use, and misuse, of the features and glitches of our implementation. 🙂
Many thanks to the team, especially to Roland Graf, Pallavi Benawri, Amy Whitesall, and Besty Howell.
UPDATE: Two of our papers at CHI 2020 received Best Paper Awards. Wohoo!
We are happy to announce that we have three papers conditionally accepted at CHI 2020 featuring our work on MRAT, our toolkit for recording user interaction sessions with AR/VR applications created with Unity and producing analytics that can be inspected in a dashboard or visualized in mixed reality in-situ; XRDirector, a new collaborative immersive authoring system that adapts roles from filmmaking to coordinate multiple co-located designers, with some of them working in VR and others in AR; and an interview study on key barriers to entry for AR/VR content creators with different backgrounds and levels of expertise, to guide the design of future tools specifically for end-user programming of AR/VR experiences.
M. Nebeling, M. Speicher, X. Wang, S. Rajaram, B.D. Hall, Z. Xie, A. Raistrick, M. Aebersold, E.G. Happ, J. Wang, Y. Sun, L. Zhang, L. Ramsier, R. Kulkarni:MRAT: The Mixed Reality Analytics Toolkit. In Proc. CHI 2020. Best Paper Award
