Case Study: Enabling Users to Easily Interact with AR Apps with SparkAR
A. Let's Start with The Why?
Augmented Reality has been around for a while, but it wasn't until the introduction of smartphones and digital apps that truly drives AR to take-off and accessible to the masses. However, underneath the curtain, AR creation experience used to be both daunting and expensive process that requires specialized knowledge and tools, which hinders designers and developers adoption. Here's one of the demo that I wrote on Swift to build an indoor POI navigation AR app known as Carijalan (Wayfinder in English)
Personally, I found the development process fun, but I do understand that for most people, it seems daunting due to complicated-looking code with high learning curve. Therefore, the very reason why I write this story is to share alternative approaches that requires less to zero technical expertise to create, publish and share AR apps quickly, especially for designers who are looking to broaden their portfolio beyond 2D applications
However, you may wonder why should I learn AR? what's the problem that require such technology in the first place?. Well, that's a pretty valid question, which is why you rarely heard about this space, compared to the likes of Artificial Intelligence, Software Engineering, Product Design and others. As a designer, I personally see that Augmented Reality used to have high barrier of device entry, which means only certain devices that may be able to operate in it's full potential, which caused adoption issues amongst general users. In some cases, application package size also becomes an issue for mid to low-end devices. Therefore, asking general users to manually download third party AR apps isn't ideal, until the arrival of social media filter such as Instagram Filter, which changed the AR landscape. Why and how?. Before I explain in details, take a look at my examples below!
My works reached thousands of audiences without spending a dime on marketing budget and simple AR development process. You can try one of my AR filter here. Now you see where I'm going with this right? 😄
Therefore, the answer to why and how social media filters changed the AR landscape is that AR can now truly take off to help shaped the future of digital marketing, through social media filters that directly connected to thousands and even millions of users without the need to download and install separate apps. To me, this is a technological revolution that most overlooked, yet powerful enough for brands and small business to seize and conquer millions of user's attention, without overspending
B. Let's Continue with The How?
First, let's create a new project. In this case, I'm using SparkAR, which is Meta's AR Development tool that we will use to publish the project directly to Instagram or Facebook. Then, as shown in the image below, there are several components that we will learn to create this Topeng Nusantara (also known as Cultural Masks in English) to spread awareness about Indonesia's cultural diversity. Let's begin!
1. Canvas and Background Segmentation → Background segmentation allows you to isolate your subject (in this case, the masks asset) from the background in real-time. This helps designers to create more immersive augmented reality experiences by making the AR content appear more natural and seamlessly integrated with the real world. Furthermore, without proper background segmentation, the AR content may appear disjointed or out of place, which can detract from the overall user experience.
2. Add Face Tracker capability → In Swift, we have Vision framework to activate face recognition capabilities, but it took several steps to implement that capabilities, whilst in SparkAR, you can toggle the feature instantly. Take a look at the example below to see the differences between implementing vision and toggling Face Tracker in SparkAR, which really shows the democratization aspect of AR technology
3. Add Materials and Textures for the Mask Assets → Materials are an essential component to determine the appearance of 3D objects in an AR scene. Therefore, since the 3D object here is the mask variants, then I prepare materials and textures for each mask variants. Some of you readers may ask where did I get the mask images?. Well, Google's your friend. However, originally, each images has different size and resolution. Thus, as a designer, I standardize all mask sizes by creating masked layer in Figma, then using tinyPNG to optimize asset size prior adding to the SparkAR file
4. Add Option Picker (Go Extra Mile) → You don't have to do this, but since I love pushing things forward, I was wondering how to create an option picker so that users can have better experience to choose different sets of masks. To see the code, you can tap here to use it in your project ✨
5. Time to publish → It's time to publish the AR project on Instagram. You can click SparkAR Publish menu in the left menu options, or access Meta Spark AR Hub then upload your project file and fill out other supporting information such as project title, demo video and others
C. Learnings
As mentioned previously, AR development used to be a daunting process and only accessible to a handful of people with specialized knowledge and technical skills. But today, that's not the case anymore with the arrival of tools such as SparkAR, Tiktok House Effect and many more, creating AR has become much easier and accessible to everyone, including product designers.
Furthrmore, for sure that there are other cases where AR can be implemented as well, such as education and enterprise (take a look at Hololens), but the reason why I specifically pick digital marketing is because it helps me to reach new users at scale (Google coined this term as Next Billion Users) which is helpful to spread awareness and drive users to learn how to adopt the technology, without having to face device constraints.