Using Augmented Reality for Brain Education: An Art and Design Neuroscience Collaboration
tldr;
A technology based education tool that uses augmented reality and facial recognition to overlay a facsimile of the human brain on the user’s head created, designed, and coded by me.
Experienced and user tested by over 2000 students and educators at neuroscience events across the US.
The application runs on a Microsoft Kinect and uses a mobile interface to control the experience.
MyBrain! is a mobile application that uses augmented reality to teach the functions and processes of the brain in relation to everyday stimuli. The theme of the project was generated from a need for interdisciplinary collaboration on the WSU Vancouver campus between science oriented curriculum and art/design programs.
The core team sought out and partnered with authorities in the neuroscience department to create a kinesthetic and engaging educational app that satisfied common core curriculum for brain education in an engaging and technologically forward way.
“Effective integration of technology, art projects and activities into outreach efforts increases engagement, makes learning personally relevant, and allows students to explore concepts by creating objects and images they can share with family and friends.”Jeff Leake
Augmented reality company Metaio released a beta version of a face modeling SDK in October 2014 that could be programmed with the C++ coding language.
A technological marriage of the public and open source Kinect SDK and the Metaio SDK combined with some rigorous hand coding allowed for a working face modeling program to accurately determine the translation and rotation of the human head and overlay a 3D model of the brain. The brain was then animated displaying basic representations of the neurological activity a person experiences when exposed to common stimuli.
Early Mockups
Initial concepts imagined a tablet based solution with a futuristic interface (think Iron Man) that would allow the user control over the brain in the viewport. Control potentially allowed for manipulating the brain based on emotion, physical stimuli, age, and more.
Beta Brain Testing
The initial MyBrain! prototype was developed to run from a desktop computer and a Kinect camera. Interaction occurred via a touch screen. The interface was simplified to just core buttons and extraneous details were stripped away.
Users were heavily engaged in the material and were interested in how they might use the technology for school. Students interacted with the program by forming connections between brain functions and the location of brain parts in their own craniums.
Educational staff that tried out the prototype were also very engaged and desired the program to begin implementing augmented reality into their curriculum immediately after testing. The program demonstrated significant promise.
Tablet Integration
Next steps included creating the mobile prototype for individual control. This allowed users to turn away from the camera and see a full 360° view of the cranium. The interface remained simplified to optimize the viewport as individuals moved and rotated their heads during use.
Continued User Testing and Research
During the beta period over 1000 users tested and experienced the application including events at the Oregon Museum of Science and Industry and the Phillips Collection in Washington D.C.
Qualitative feedback and usability issues were gathered to improve the application and experience in an educational setting.
“Brittany Wouden brought along her sought after educational myBrain! app, which attracted visitors delighted to see an image of a brain superimposed on live video of their own head! They dramatically swung open hemispheres, and gamely explored which networks are active during various cognitive states…”NWNoggin
Companion Website: MyNoggin
MyNoggin! is a companion website and research tool to MyBrain! The tool was used to gauge learning retention of students that used MyBrain! as part of their neuroscience curriculum. The website was comprised of quizzes that rewarded badges upon successful completion. These badges allowed educators to measure learning retention over time of the students participating.
Analytical data was also collected to measure when, where, and how often students were studying and completing quizzes.
Designs completed by me in collaboration with NWNoggin. The client desired something simple, colorful and fun. The website was meant to be flat and encompass the feeling of art in an educational setting.
