Rapid Physical-Digital Prototyping with ProtoAR

We’ve just finished shooting a video summary illustrating some of the main features available in ProtoAR, our rapid prototyping tool for mobile augmented reality interfaces.

We created ProtoAR specifically for interaction designers. There are two key innovations in ProtoAR: (1) cross-device multi-layer authoring tools for live editing of mobile AR apps on phones; (2) interactive capture tools to generate mobile screens or AR overlays from paper sketches, and 3D models from Play-Doh.

We look forward to presenting the paper and more results at CHI 2018 in Montreal this April, and we have some more plans with ProtoAR. So stay tuned! For now, here is a preview of what one can do with ProtoAR:

Postdoc/PhD Student Positions

The Information Interaction Lab at the University of Michigan School of Information has the following openings for Postdocs and PhD students:

  • one opening for a Postdoc (ideally, with a strong publication record in premier HCI conferences and experience with AR/VR technologies) starting 2018
  • between one and two openings for PhD students (with experience in interactive systems and devices including, but not limited to, AR/VR technologies) starting Fall 2018 (applications are due 1 December 2017).

Our lab is a rapidly growing group of talented researchers and students working on a mission to create and study next-generation user interfaces that make it easier to work with information and technology in more natural and powerful ways. We are working to push the state of the art in interface software and technologies, most recently focused on AR/VR, but also including voice-based assistants like Alexa, wearable devices such as smartwatches, and cross-device interfaces.

Be sure to check out our research page. You can find more information on how to apply on the Jobs & Internships page.

Fall 2017/Winter 2018 Research Assistant Positions

Update: I have recently hired 5 new students and I’m not actively recruiting new students for Fall 2017/Winter 2018, but I’m always happy to hear from interested students, especially anyone with experience in AR/VR interfaces.

The Information Interaction Lab at the University of Michigan School of Information is looking for two master’s students to assist with a range of ongoing AR/VR projects. After a short trial period, these positions can be converted into paid temporary research assistant positions (pay commensurate with experience).

We are looking for students with solid AR/VR design and development experience. Ideally, you consider yourself an expert in Unity and/or Three.js/A-Frame, and have experience working with AR frameworks including HoloToolkit, Tango, ARCore, ARKit, AR.js, etc. We realize that there is only a tiny fraction of students out there that have such a background, so a compromise might be that you can at least demonstrate significant front-end design and development experience, and the potential to quickly navigate the AR/VR space and learn a range of new technologies as required for our ongoing research projects.

If you think that could be you, please send an email to Michael Nebeling. Make sure to include a short paragraph about yourself and your interests in working with us. If you have it ready, please also include a CV.

Fall 2017 Hackathons

Update: Thank you to everyone who attended one or more of our special design jams on augmented reality in September! It was great that so many of you could make it. Feel free to join our MCommunity list for updates on future design jams.

As we are getting ready for the next semester, I wanted to announce here that I will again host regular student design jams (or hackathons) in my research lab starting in September 2017.

Hackathons are 3 hour blocks (most likely on Mondays or Fridays 2-5pm) for students to work on user interface research and design challenges, and this is typically done in teams (no need to come in teams, we will formulate teams at the hackathons). Sometimes design challenges continue over multiple weeks, and it happens rather frequently that students end up doing a research project with me based on some of the initial hackathon ideas.

Previous hackathons revolved around voice-based assistants using Amazon Echo and Google Home, multi-modal interaction using Kinect and Leap Motion, and cross-device interfaces using the lab’s own technologies. For the next semester, there will be new design challenges around these same topics, but there will also be a new emphasis on augmented reality interfaces.

The lab has recently obtained 2 Tango AR phones and 4 Microsoft HoloLens AR headsets that students will have access to and will be encouraged to use in projects. Again, we also have new technologies developed in the lab’s research, and my goal is to drive these technologies forward with the help of students and these hackathons.

Mi2 Lab at CHI 2017

Professor Michael Nebeling, Postdoc Max Speicher, and incoming PhD student Brian Hall attended CHI 2017, adding to the strong presence of UMSI at the premier HCI conference this year in Denver, Colorado.

Michael participated in the HCI.Tools workshop with leading technical HCI researchers discussing the opportunities and challenges that come with systems and toolkits research. In preparation for the workshop, Michael put together a position paper, “Playing the Tricky Game of Toolkits Research”, in which he summarizes the challenges he faced when starting out as a new technical HCI researcher and the lessons he has learned over the years from his own research program as well as participating in program committees at ACM CHI and EICS.

Michael also presented his new paper, “XDBrowser 2.0: Semi-Automatic Generation of Cross-Device Interfaces”, which describes two studies informing the design of a next-generation web browser able to distribute existing web interfaces between two or more devices, including tablets, smartphones, and smartwatches. In his presentation, Michael started out by saying that he was actually hoping to build a new system that can automatically generate cross-device interfaces, but that he soon realized that more research was required, leading to the two new studies described in his paper. Michael also acknowledged the support from his former BSI/MSI students and research assistants, Andie Dumas, Annabel Weiner, and Licheng Zhu.

A particular highlight for the lab was Brian’s participation in the CHI Student Research Competition. The competition involved multiple stages starting out with poster presentations on Day 1. On Day 2, a smaller group of selected students invited to the next stage were given the opportunity to summarize their research to a larger audience in a formal presentation. Finally, the top 3 students selected by leading researchers in the HCI community were announced at the end of the CHI conference in front of everyone.

Brian presented the first study he carried out as part of his project on investigating usable and effective interfaces for controlling drones. We were very proud of Brian who was recently awarded the prestigious NSF fellowship and now successfully made it through all stages and finally placed third in the highly competitive CHI Student Research Competition in the undergraduate category. Congratulations! We are looking forward to having Brian join us this Fall!

Michael Nebeling Co-PI on ESSI Grant

My colleagues, Steve Oney and Sun Young Park, and I comprise one of four research teams chosen to receive a $200,000 pilot grant from the University of Michigan’s Exercise & Sport Science Initiative (ESSI). Launched in 2016, ESSI draws on the expertise of faculty from a wide range of disciplines across campus, Michigan Athletics and industry partners to optimize performance and health for people of all ages and abilities.

We proposed a plan to build a “data warehouse” that would enable recreational athletes, coaches and even fans to collect physical data from various sources, such as FitBits, smartwatches and phones in order to improve their performance. The Information Interaction Lab will play a key role in developing new interfaces and interaction techniques using augmented reality to empower those end-users.

Read more…


Brian Hall joining as PhD Student

We’re happy to announce that Brian Hall will be joining the UMSI PhD program and our research lab in September. He will be advised by Professors Michael Nebeling and Mark Ackerman. Brian is joining us from the University of Wisconsin Stevens Point. Brian brings significant experience working as a software developer and undergraduate researcher for several years. Most recently, he was visiting with Laura Dabbish in the Human-Computer Interaction Institute at CMU. We’re really looking forward to him joining our lab and the UMSI community!

Max Speicher joining as Research Fellow

We’re happy to announce that Max Speicher has joined UMSI as a Research Fellow for his postdoctoral research in our lab. Max brings several years of industrial research experience and expertise in the AR/VR space with him. Most recently, he was working  at HoloBuilder. Max will participate in a U-M wide research initiative in AR/VR and help us establish the lab in that area. Stay tuned!