Finally, A Way To Do Face Tracking With FaceID

For a while now I’ve been looking into different ways to do motion capture on the cheap. You won’t believe how ridiculously expensive it is to create even an at-home setup that captures your body and face movements for animation software. That was until I came across a link on twitter that changed everything.

When I was at university I was looking for a way to incorporate cotton capture into the animation module of the course as a way to gain a few extra points in the modules project. Unfortunately, I found out that it was bloody expensive and never got to do anything along these lines. I did come across RADiCAL, it an app for your phone that uses the camera to record your bodies movement and translate it into movement data for your software of choice. Great I thought… until I saw the price £8/month would allow you to export just 3 mins worth of movement. They do have other plans but still if your experimenting with the software the cost will soon add up.

I also came across a few open-source projects for blender that would allow you to do face tracking to but they either required you to put dots on your face with a sharpie (not going to happen) or use 8 different bits of software to get something useable. Then I got the link on twitter.

Epic released an iOS App called Live Link Face that allows you to simply point your phone at your face and (once linked) the tracking info from the True-Depth front-facing cameras (on the iPhone X and above) & VRKit is sent to the unreal engine where the face is animated in real-time matching your movements.

As you can imagine I am really excited to play with this and see what I can do. I have two problems though. I won’t be able to do anything until I get my new phone in a week or so and I’m going to have to learn how to create the mapping texture and add it to the characters I can download at Mixamo.

I’m actually looking forward to learning how to put the two together as well as creating my own characters too, but I know for a fact that even if I pick up the technique it will still take some time to iron out the creases and create something that is suitable to be shared with others.

I’m sure I’ll be posting a video even now and then as I get my head around the initial setup and then as I progress to creating my own characters. I’m looking forward to the challenge of what I’m sure will become one of my weekend projects.

What would you create with the new iOS app for the unreal engine? can you recommend any easy to use free body tracking software? Let me know in the comments below.