- Time: Aug 2018 – Dec 2018 (Fall 2018 Semester)
- Place: Entertainment Technology Center
- My Role: Programmer / 3D Modeler / Sound Designer
- Platform: Custom Pass-through VR Rig (VR HMD+ZED Mini+Leap Motion)
- Engine: Unity
- Output: Design Documentation, Four Fully Functional Prototypes
- Website: http://www.etc.cmu.edu/projects/pupil/
Pupil: Augmented Learning is a discovery project focusing on the design pattern of Augmented Reality (AR) application utilization in future classroom education.
After researching we find that current All-in-one AR goggles, including Microsoft HoloLens, Magic Leap One and more, does not meet our expectation on interactivity. Meanwhile, ZED Mini with VR goggles provide the best pass-through VR experience today to simulate AR and we make it compatible with Leap Motion, the state-of-the-art skeletal hand tracking system. We favor the crisp interactivity to portability for our platform choice.
At the end of the semester, we have crafted four interactive demos with different focuses and subjects. I participated in various roles during the development process and the following is a break down of my contribution.
I designed and programmed this simple interactive triangle prototype. Guests can move vertices freely and look real-time angle measurement on the hand panel showed above. After playtesting, we got a lot of feedback on the physical strain of using the hand panel. I later changed the angle measurement tool to display on the side of each vertex, freeing guest from using the hand panel and also display angle measurement aside where the angle locates.
You can find more of this demo here.
Coordinate System Demo
This 3D- Coordinate System demo is developed at the same time when I implemented the Triangle Demo by another programmer in the team. Later, I added sound effects to the demo to gain more tactile feedback when guests interact with it. You can find more of this demo here.
Rainforest Biome Demo
This is the third demo we have developed to explore more interaction pattern suitable for AR + hand-tracking system. I did 3D Modelling of organisms in this demo and assisted with programming. I also added sound effects to this demo. Here you can find more detail about it.
Multi-user Chemistry Demo
In our last demo, we explored multi-user application with shared-space collaboration functionality. I helped programming on user experience including implementing sound effects, visual feedback and more. You can find more here.