Rapid Physical-Digital Prototyping with ProtoAR

We’ve just finished shooting a video summary illustrating some of the main features available in ProtoAR, our rapid prototyping tool for mobile augmented reality interfaces.

We created ProtoAR specifically for interaction designers. There are two key innovations in ProtoAR: (1) cross-device multi-layer authoring tools for live editing of mobile AR apps on phones; (2) interactive capture tools to generate mobile screens or AR overlays from paper sketches, and 3D models from Play-Doh. These are described in detail in our CHI 2018 paper.

We look forward to presenting the work at CHI 2018 in Montreal this April, and we have some more plans with ProtoAR. So stay tuned! For now, here is a preview of what one can do with ProtoAR:

Fall 2017/Winter 2018 Research Assistant Positions

Update: I have recently hired 5 new students and I’m not actively recruiting new students for Fall 2017/Winter 2018, but I’m always happy to hear from interested students, especially anyone with experience in AR/VR interfaces.

The Information Interaction Lab at the University of Michigan School of Information is looking for two master’s students to assist with a range of ongoing AR/VR projects. After a short trial period, these positions can be converted into paid temporary research assistant positions (pay commensurate with experience).

We are looking for students with solid AR/VR design and development experience. Ideally, you consider yourself an expert in Unity and/or Three.js/A-Frame, and have experience working with AR frameworks including HoloToolkit, Tango, ARCore, ARKit, AR.js, etc. We realize that there is only a tiny fraction of students out there that have such a background, so a compromise might be that you can at least demonstrate significant front-end design and development experience, and the potential to quickly navigate the AR/VR space and learn a range of new technologies as required for our ongoing research projects.

If you think that could be you, please send an email to Michael Nebeling. Make sure to include a short paragraph about yourself and your interests in working with us. If you have it ready, please also include a CV.

Fall 2017 Hackathons

Update: Thank you to everyone who attended one or more of our special design jams on augmented reality in September! It was great that so many of you could make it. Feel free to join our MCommunity list for updates on future design jams.

As we are getting ready for the next semester, I wanted to announce here that I will again host regular student design jams (or hackathons) in my research lab starting in September 2017.

Hackathons are 3 hour blocks (most likely on Mondays or Fridays 2-5pm) for students to work on user interface research and design challenges, and this is typically done in teams (no need to come in teams, we will formulate teams at the hackathons). Sometimes design challenges continue over multiple weeks, and it happens rather frequently that students end up doing a research project with me based on some of the initial hackathon ideas.

Previous hackathons revolved around voice-based assistants using Amazon Echo and Google Home, multi-modal interaction using Kinect and Leap Motion, and cross-device interfaces using the lab’s own technologies. For the next semester, there will be new design challenges around these same topics, but there will also be a new emphasis on augmented reality interfaces.

The lab has recently obtained 2 Tango AR phones and 4 Microsoft HoloLens AR headsets that students will have access to and will be encouraged to use in projects. Again, we also have new technologies developed in the lab’s research, and my goal is to drive these technologies forward with the help of students and these hackathons.

Winter 2017 Comerica Hackathons

Update: We’ve changed the format by doing 2 hackathons: one leading to a paper prototype and the other leading to a digital prototype. We’ve decided on two consecutive Thursdays: March 30 and April 6.

Our lab will host and run weekly 3-hour hackathons in collaboration with Comerica‘s UX team. Their team is setting us a design challenge around improving the mobile banking experience. The hackathons will be led by Professor Michael Nebeling and his team from the Information Interaction Lab at UMSI, and will provide BSI and MSI students an opportunity to apply and deepen their interaction design skills.

The hackathons will start in mid-January 2017, and will stretch over a period of 8-9 weeks. There will be plenty of opportunity to engage with Comerica’s growing UX team, and we are also planning a larger final presentation to Comerica (which may include a sponsored trip) at the end of the project.

We are currently looking for students that are interested in participating in these hackathons. We require that students are committed to this project and are prepared to volunteer 3 hours of UX design and research work in our lab and 1-2 hours of homework per week. We also make it a requirement that students already have a basic understanding of interaction design, HCI principles and methods, which can be demonstrated based on successful completion of our SI 482 and SI 582 interaction design courses, or significant project work completed in the context of external projects and internships.

Please send an email directly to Professor Nebeling at nebeling@umich.edu to express your interest and find out more. We will be accepting 10-15 students into this project.

First Semester Done

Photo from our last hackathon in December, showing a number of students involved in the lab, picture taken with Kinect Photo

Fall 2016 was our first semester at UMSI, and a number of things happened:

  • set up our new lab space
  • purchased an initial set of wearable, mobile, and stationary devices: five 4K displays, a number of smartwatches, mobile phones, Kinects, portable projector, Surface Book and Surface Studio (pre-order)
  • got our lab’s logo designed
  • primarily worked on interfaces to support our lab activities, but also initial research

Our projects included:

  • lab backend and platform that manages devices and users in the lab
  • lab dashboard, an ambient interface pulling together information on all kinds of lab activities
  • Slack and Trello integrations
  • Alexa voice interfaces for the lab
  • Kinect gesture, voice and multimodal interfaces for the lab based on Kinect Analysis
  • sketching and note-taking interfaces
  • continued our research on the XDBrowser project:
    • support for semi-automatic distribution (upcoming CHI’17 paper)
    • round smartwatch interface
    • projector-based interface
    • in-car interface

Other activities included:

  • presented research overview at MISC seminar
  • taught SI 482 Interaction Design Studio
  • organized weekly hackathons for interested students
  • held cross-device tutorial at the International Conference on Surfaces and Space (ISS’16)

Fall 2016 Weekly Hackathons

Our lab will have weekly hackathons on Thursdays, 3pm-6pm.

Our hackathons will be completed in 3 hours (they are still “real” hackathons, we’re just faster 😛). We will start with introductions and goals—what’s the goal you want to accomplish over the next 3 hours? Goals do not have to be “researchy” and lab-related, they can be personal goals as well (e.g., implement functionality X, design usability study for project Y, or finish assignment for course Z). Sometimes a hackathon may be dedicated to a particular goal relevant to the lab. But even then, you may come and choose to contribute to that goal or work on your own. The main thing is to come together as a group and feel productive working towards the announced goal, be it individually or in teams.

Important note: these hackathons, although organized by the Information Interaction Lab, are not restricted to lab members. We encourage everyone who is affiliated with UMSI (faculty, staff, students) and interested to join us and have some fun (plus getting some work done, too!).

If it’s going to be your first hackathon, please contact us to let us know you’ll be attending and find out about the exact location.