THE BOX AR

The Concept

THE BOX concept originated in December of 2015, and its first debut was designed and developed as a VR experience to be released with the launch of the Samsung Gear VR. The core experience builds upon the universal thrill of opening a gift, but in a virtual format. Each gift or BOX delivers a whimsical Augmented Reality experience in the form of collectable characters, delightful dioramas, interactive pets, art tools, environmental transformations, and portals to exotic worlds. AR experiences are collected and stored in a library for later play.

The Task

Based on the positive feedback from THE BOX VR, combined with our successful ARKit demo that received more than 125k views on YouTube in just under two weeks, we saw a market opportunity for porting THE BOX concept to mobile AR. With the highly anticipated launch of ARKit for iOS 11, we decided to adapt THE BOX for this new platform, and partake in the exciting opportunity to deliver AR experiences to millions of mobile devices for the first time ever.

Our end goal? To spark consumer excitement around Augmented Reality, provide incentive for continual engagement, and drive conversation around the potential of mobile AR.  

Our Approach

As we began to think about design for the interface, content, and experience, we wanted to make sure that every part of the end-to-end journey was optimized for the mobile AR novice. To reach the broadest audience we knew it would be critical to strike a balance between simplicity and instructional support.

Interface Design

Given that mobile devices can be used just about anywhere, our design decisions needed to accommodate variable viewing environments. Our app’s color palette was specifically designed to be viewed in light and dark environments, as well as in natural and artificial lighting. To welcome all ages and speakers of all languages we designed our interface with heavy reliance on iconography.

User Experience

In these early days of designing for Augmented Reality, even the simplest interactions must be carefully considered. Here are a few key caveats we kept in mind while designing the experience:

• Our users likely haven’t had many encounters with Augmented Reality, thus it is of utmost importance to guide them through the experience while being careful to not make the instructional support feel burdensome
• AR is in its infancy, we must continuously test user behavior and connect to a greater network for insights on user experience and interactive design
• New technology = high risk for app abandonment if the experience is not naturally intuitive and fun

At the core of our strategy, we leaned on best practices from Apple’s Human Interface Guidelines, leveraged new recommendations for designing in AR, and conducted ongoing user testing. We worked hard to create a set of rules for placement of and interaction with the augmented content, as well as to define aspects of user interaction in AR that would add the greatest value to experiences.

For new users of the app, we incorporated a short introductory tutorial. Brief walkthroughs for each content type were delivered by way of a supplemental information screen with each new box. Dependable user interactions were established by delivering content with a consistent set of features and interactive elements. Constraints and specifications were developed not only with the user in mind, but also to support rapid content development, such as our sizing specifications for the arrival of new box items.

Content Creation

We started with designing six core content types consisting of:

Portals – Experiential worlds that are placed in a physical environment and entered through an augmented window to view a 360 degree scene. These can be used to immerse a user with 360 imagery or 360 video from far away lands, to exhibit artwork, or to tell a brand story.

Environmental Enhancers – These boxes bring natural elements and ambient effects to any physical space. They alter reality in delightful ways and enhance emotion in a moment in time.

Dioramas – Curated, animated 3D scenes that tell all types of tales. Dioramas can be viewed from above or expanded to life size, allowing the user to become part of the story.

Animated Objects – Interactive 3D objects are designed to be placed and played with in the physical world. These objects can be assembled to create custom scenes, tell a story, punk a friend, or to simply embellish everyday reality.

Art Tools – Creativity is unlocked in an entirely new dimension when users get to embellish a physical space by drawing, decorating, and adorning it with Augmented Reality. These art tools can be leveraged to create custom art, write AR messages to send to friends or share on social media, or even to annotate a 3D environment.

Cutout Boards – Simple in concept, but effective in interactivity, cutout boards give users a virtual cutout image for people to pose behind. They are a great device for entertainment and have the potential to drive fun marketing messages.

Content Pipeline

Anyone in the industry will know that 3D content creation is time consuming and expensive. Given that THE BOX AR promised daily AR content delivery, we were challenged to quickly optimize our content pipeline for efficiency and scale.

Establishing Shortcuts

Shortly after launching THE BOX AR, the challenges of rapid content production became apparent. We immediately went to work to tweak our process with the goal of making the general aspects of the content production pipeline as straightforward as possible to open up more room for polish. As we learned more about ARKit and Xcode, we were able to establish a list of handy shortcuts and sidesteps to common problems. By the end of the project, it had allowed us to push the envelope of complexity with our assets.

 

Adaptability

Due to the demanding daily deadlines for content production we often leveraged existing assets from various websites, such as the Unity Asset Store and Turbosquid. The trade-off was a perpetual unpredictability of what to expect from each individual asset. Often, a model would have the wrong scale, flipped normals, missing UVs, corrupt animations, or a combination thereof. It became normal for us to anticipate a certain level of chaos. This, in turn, paved the way for a number of catch-all problem-solving techniques. 

For improper scaling, we made a practice of freezing transforms before exporting to accommodate resizing. For missing UVs, re-exporting and re-importing as an .obj format reliably fixed the issue. For problematic animations, we made good use of stepped interpolation where necessary to bypass flipping issues. And the best general solutions for conserving file size with assets were to (a) remove any joints that weren’t needed, such as fingers or hair, and (b) shortening the animation loops to double-speed, effectively halving the number of keyframes needed to store the exported animation, and then slowing them down by half again once they were imported into Xcode. The whole experience was a very new and interesting method of ensuring quality and efficiency.

Development

For this project, we decided to veer outside of our Unity comfort zone and develop natively, with Apple’s Xcode IDE and its SceneKit and ARKit framework. While both platforms have their pros and cons, we ultimately decided to go native because of the newness of ARKit APIs and the fact that the Unity ARKit plugin had not had a chance to be thoroughly proven before the launch of iOS11.  We found that it was a great approach for THE BOX AR as the build time was an order of magnitude faster and it was an opportunity to test the limits of what ARKit can natively do on iOS11.

One of the biggest challenges with native iOS11 development is that it requires all 3D assets to be in the COLLADA/DAE format, rather than the more standard FBX format used in Unity. Because we leveraged a good deal of existing FBX assets to accommodate the aggressive content production requirements, the file conversion process became a key part of the overall development and it needed to be optimized for scale and efficiency. The pipeline development required close collaboration and constant iteration between our designers, animators and developers, and a good deal of creative problem solving and engineering, before we had it dialed in.

 

 

What’s Next

From brand marketing to 360 video tours to product visualization, business opportunities for THE BOX AR are endless. 

“THE BOX is our vision for the future of marketing in Augmented Reality,” says 8ninths CEO Adam Sheppard. “This novel ‘Short Form AR’ format has the potential to become a daily way for users to receive mind blowing new content and share with others. We think that brands will be excited to be able to connect with their audience through this novel format in a way they’ve never been able to before. These brief escapes from the real world are almost like a daydream, changing your mood and environment for a short time — without having to wear a headset.”

If you’re interesting in discussing how THE BOX AR framework or Mobile AR could be used for your business, contact us with the form below and we’ll setup a time to discuss your business needs.