Following up on our preview at Facebook Connect, today we’re rolling out new hand and body tracking capabilities as part of Spark AR Studio version 126. We’ve already seen some really fun, early examples of these tracking capabilities in our beta tests, and we’re excited by the possibilities that hand and body tracking opens up for creators interested in more performative AR effects, especially for Instagram Stories and Reels. Let’s dive in and explore these new trackers:

2D body tracking

@instagram, @positlabs, @finofinissimo

Using the new 2D body tracking capabilities, you can now anchor effects to parts of the body, like arms and legs, and up to 20 different key points (or joints), like elbows or knees. And your effects can be applied to a single person, multiple people, or to partial parts of a body in a scene.

To help make effect creation easier, we’re also introducing new body tracking patches. How you use them will depend on which key points of the body you want to track:

1. Body landmark patches track specific parts of the user’s body, from head to toe. For example, you could use these patches to create line trail effects where particles emit from certain joints.
2. Body bounding box patch tracks the top and bottom corners of the user’s body creating an invisible rectangle, or bounding box. You could use this patch to create, for example, a body swapping effect.

Additionally, to help you build effects faster, we’ve developed a new body tracking template called “Body Movement.” This new template is now available in Spark AR Studio.

We hope you’re excited to try out body tracking, we think it has the potential to open up more complex and imaginative effect types, ranging from full-body characters to gravity-defying dance moves. And we’re already working hard to extend today’s 2D capabilities to 3D key points, we’ll have more to share on this soon.

Hand tracking

@boonjun1003, @tokyyto, @enuriru, @alwayscodingsomething, @instagram

As part of today’s Spark AR Studio update, we’re releasing new hand tracking capabilities that make it easier for you to create effects that respond to a user's hands. There are two primary ways to create effects that interact with hands:

1. By adding a hand tracker object, any object(s) added to your scene as a child of the hand tracker will move with the user’s hand. Only one hand can be tracked in this way.
2. By adding a hand bounding box patch, which will plot an invisible rectangle around a selected hand, calculating 2D coordinates for all four corners and the center of the rectangle, as well as its height and width. Both hands can be tracked when using this method.

Be sure to check out our new hand tracking tutorial which demonstrates how to create an effect that zooms in each time a hand is found in the camera. We also have a new template available in Spark AR Studio today called “Hand Movement,” that can be used to help jumpstart your next idea for hand effects.

Eventually, we plan to support a robust set of hand keypoints and advanced gesture controls (e.g., fingers and wrist), to give you more flexibility and precision for placement. We’re also building interoperability with other capabilities (e.g., body tracking), and testing across lighting conditions, to ensure the best possible experience for all users.

Start building your next AR effect

We’re excited to invite all creators to participate in the upcoming XR Hackathon. Register today to be among the first to receive details when the Spark AR challenges are announced, and compete for your chance to win a total of $700,000 USD in cash prizes.

Regardless of whether you’re competing in a hackathon or simply exploring new ideas for self expression, we’re excited to see what you’ll create with these new capabilities. And if you haven’t already, we hope you’ll join the Spark AR Community soon to show us what you’re working on!

Subscribe to the Meta Spark Blog