Sending a voodoo-masked or pig-nosed selfie to your friends through Instagram, Facebook or Snapchat is one of the little joys of life. And isn’t it awesome that you can just use your camera lens to leverage this advanced technology. Just like emojis, face filters are fun even for someone other than just young people.
But how about building your own face filter applications which uses augmented reality (AR) technology to superimpose your own created face masks, doggie tongues or some other animal masks on your own or your victims face. Digitally of course.
In this beginner AR development guide we are going to be using creative tools in Unity and build a face tracking / face filter app on your desktop. If you go about selling instagram face filters for big money - hey, give us a shoutout.
If you want to create face filters for Snapchat, you should use their dedicated web app — Lens Studio. Snapchat and Instagram are also a good source for inspiration. Check out AR Filters from Lenslist (my favorite are: tiger, deer, and these crazy eyebrows). Embrace your wild side!
What are AR Face Filters?
Face filters are snapchat like designs that overlay on top of a tracked face. They’re useful for seeing how a pair of glasses look or what hairstyles would look best or even detecting emotional queues.
We’ll be building some of our own Snapchat or Instagram-like filters in Unity.
Get the Assets
Get the Tutorial Project Files and Assets:
Download FaceFilters Unity package here
Video Walkthrough:
On-Demand AR Face Tracking and Face Filters Workshop
Want to follow along with the instructor? Submit the form and watch the video tutorial on-demand here.
Unity Setup
Create a project using Unity 2020 ↗.
In the Package manager of unity (Windows > package manager) download and install AR Foundation, AR Core XR Plugin (if using Android) and/or AR Kit XR Plugin (if you’re building for iOS devices).
We’ll need these to enable AR on our mobile devices so it can use phone lenses to superimpose the AR picture in the real world.
Prepare the Scene
In your project, add the following folders:
Materials
Prefabs
Scripts
Textures
Drag your pictures for your face filters into the textures folder and create Cutout materials from them. In your materials folder create a material for each texture you have and set the rendering mode to Cutout.
Next let's get the scene ready. To the hierarchy, add an AR Session Origin and and AR session as separate gameobjects.
Next we’ll create a canvas with a button to toggle between the face materials!
Create a canvas and then create a button on your canvas.
We’ll be building the face prefab next. Create an empty gameobject in the hierarchy. Add a mesh filter, mesh renderer, AR Face, and AR Face Mesh Visualizer to the empty gameobject.
Make this into a prefab by dragging and dropping it into the Prefabs folder. You can now delete the gameobject from the hierarchy.
On the AR Session Origin, add an AR Face Manager script by clicking on Add Component and searching for AR Face Manager. Now you can drag and drop the face prefab we’ve just created into where it says Face Prefab.
Next we’ll get into scripting!
Code Interactions
Create a new script called FaceSwap and paste the following code in:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
public class FaceSwap : MonoBehaviour
{
public List<Material> faceMaterials = new List<Material>();
private ARFaceManager faceManager;
private int faceMaterialIndex = 0;
void Start()
{
faceManager = GetComponent<ARFaceManager>();
}
public void SwitchFace()
{
foreach(ARFace face in faceManager.trackables)
{
face.GetComponent<Renderer>().material = faceMaterials[faceMaterialIndex];
}
faceMaterialIndex++;
if(faceMaterialIndex > faceMaterials.Count)
{
faceMaterialIndex = 0;
}
}
}
Now attach the script to the AR Session Origin. In the face materials, increase the size of the list to 7 (or how many materials you have) and drag from your materials folder into each element.
Now we have to hook up the button to the script! Click on the (+) icon for the On Click on your button. Drag the AR Session origin from the hierarchy into the None (Runtime only) field and click on the drop down menu. Select FaceSwap.SwitchFace and that’s it!
Build and Test
Now it’s time to build it onto our phones. This will be showing Android build settings.
Switch over your build settings platform to Android and look into the Player Settings. Under other settings, remove Vulkan.
Set the minimum API level to Android 7.0 ‘Nougat’ (API Level 24). Under the XR Plug-in Management tab, make sure ARCore is checked if using Android and ARKit is checked if using iOS.
Build and run it!
