As a partner of Quantum Interface, Part Time Evil designed and built an AR app for a global pharmaceutical company. The app has transformed medical device training by using augmented reality, 3D animation, gamification, and storytelling to help patients, caregivers, and healthcare providers learn how to properly operate the device. The app’s fun, interactive, and world-scale 3D experiences help users improve treatment awareness, adherence, and engagement and retain key instructional messaging. The app has now become a part of many healthcare providers’ tool kits and is used to onboard new patients and supplement existing patient training.
Objective & Challenge
Design and develop an AR app to revolutionize medical device training, catering primarily to children, caregivers, and healthcare providers. The project aimed to create an immersive and engaging medical instruction experience, focusing on children. The goal was to improve treatment awareness, adherence, and engagement among patients, caregivers, and healthcare providers. The challenge was to blend educational content seamlessly with fun, interactive, and gamified elements, ensuring effective learning.
Research
User interviews were conducted with both healthcare professionals, and parents and their children diagnosed with chronic conditions requiring medical injections to gain more insight about the user needs and pain points.
Frustration with Manuals and Complexity:
- Confusing Instructions: Parents reported difficulty understanding the written instructions in the manual. Multi-step processes while handling the physical device simultaneously led to confusion and frustration.
- Limited Visual Aids: The lack of clear and engaging visuals in the manual made it challenging for children to follow the steps.
- Inconvenient Reference: Parents found it cumbersome to constantly refer to the physical manual during the injection process.
- Short Attention Spans: Children struggled to focus on lengthy demonstrations and written instructions. Maintaining their attention throughout the entire injection process proved difficult for parents and healthcare providers.
Difficulties with In-Person Training:
- Time Constraints: Busy healthcare professionals often lacked sufficient time to provide comprehensive in-person demonstrations during consultations.
- Standardization Challenges: Inconsistencies in how healthcare providers explained the process led to confusion for parents and children.
- Limited Repetition: The inability to easily repeat demonstrations during home use made it difficult for parents to reinforce proper technique with their children.
The Solution
1. Augmented Reality and 3D Animation
Utilizing Unity3D’s AR Foundation, we integrated augmented reality and 3D animation to create world-scale experiences. This approach allowed users to learn how to operate medical devices in a realistic and interactive environment.
2. Gamification and Storytelling with Zen
To keep children engaged, we introduced Zen, a cute and lovable creature, as the face of the app. Zen guided users through the app, demonstrating device operations and delivering important messaging. The character evolved and took new forms as users progressed through lessons.
3. Quiz-Based Reward System
A quiz at the end of each lesson added a gamified element. Successfully completing quizzes earned users coins, which could be redeemed for accessories to dress up Zen or unlock alternate skins. This gamification strategy anchored the learning experience for children, making it enjoyable and rewarding.
4. Localization System
Recognizing the diverse user base, we implemented a localization system to provide custom content for specific regions such as Thailand, Hong Kong, Germany, United Kingdom, Australia, the app’s content is tailored to local needs and preferences.
5. Cross-Platform Availability
The app was developed with Unity3D for both iOS and Android platforms, ensuring accessibility for a wide range of users. This multi-platform approach allowed us to reach a broader audience, enhancing the impact of the app.
Development Process
UI/UX Design and Implementation:
In the initial stages of the project, a significant focus was placed on creating an intuitive and visually appealing user interface and user experience. Through iterative design processes, we developed a seamless and user-friendly interface that ensured easy navigation and engagement. The UI/UX implementation aimed to strike a balance between educational content and the gamified elements, creating an enjoyable learning journey for users.
AR Integration using Unity3D’s AR Foundation:
The heart of the app lies in its augmented reality (AR) features. While initially integrated with Vuforia, we eventually made the switch to AR Foundation due to the lack of resources and support from Vuforia. Additionally, Vuforia updates lagged behind in conjunction with Unity Cloud Build Automation causing issues to our pipeline. Transitioning to Unity3D’s AR Foundation was thus a more logical step since it was developed by Unity themselves. Leveraging this powerful framework, we seamlessly integrated AR technology into the app, enabling users to interact with medical devices in a realistic and immersive 3D environment. The AR integration added a layer of interactivity and engagement, transforming the learning experience into a dynamic and memorable one.
Development of Gamification Elements:
To enhance user engagement, especially among children, gamification elements were strategically incorporated. This involved creating quizzes at the end of each lesson, rewarding users with virtual coins upon successful completion. The earned coins could then be used to accessorize and customize the app’s mascot, Zen. This gamified approach not only made learning enjoyable but also provided a tangible incentive for users to progress through the app’s educational content.
Creation of Robust Localization and Data Management Systems:
Recognizing the need for tailored content in different regions, we implemented a robust localization system. This system allowed for the adaptation of the app’s content to specific cultural and linguistic preferences, ensuring relevance and resonance with users in diverse locations. Additionally, a sophisticated data management system was put in place to handle the complexity of localized content, ensuring a seamless and personalized experience for users across different regions.
Cloud Content Delivery for Dynamic Updates:
To facilitate continuous improvement and flexibility, a cloud-based content delivery system was implemented. This enabled the dynamic update of minor changes and tweaks to the app without the need for distributing a new app version. This approach allowed us to swiftly address any issues, update content, and refine the user experience in real-time, ensuring that the app remained current and responsive to user and client feedback.
Impact and Results
The app has successfully transformed medical device training, becoming an integral part of healthcare providers’ toolkits. It is widely used to onboard new patients and supplement existing patient training. The gamified approach with Zen has proven to be engaging for children, fostering improved treatment awareness and adherence. The app is currently available in select countries and is scheduled for release in more regions in the near future. The success of the initial launch paves the way for expanding its reach and impact on a global scale.