Spark AR Previews Multipeer API for AR Video Calling

The pandemic has reshaped so much of how we experience the everyday world, from where we work, to how we shop, learn, and perhaps most important, how we communicate and connect with each other. Over the last year, our team has been exploring and investing in new ways AR can help people feel a stronger sense of presence and togetherness when they’re otherwise physically apart, and we think video calling in particular, presents some exciting new opportunities.

That’s why today at F8 Refresh, we’re previewing an advancement in AR for video calling with the introduction of our Multipeer API, and we’re inviting creators to join our beta program. These new capabilities are available on Messenger now and will be opening up for Instagram and Portal soon.

While video calling has a long history, it has mostly consisted of a video channel coupled with an audio channel. Now, with the introduction of our Multipeer API, we’re introducing a data channel that will allow AR effects to communicate with each other seamlessly, across users and devices. We think this is a fundamental shift, one that will help move AR from a single user, single screen experience to a multi-user, multi-screen experience — and ultimately, unlock incredible new use cases for group AR effects.

Here are just a few conceptual examples of AR effects for video calling that we’re sharing today in the Spark AR session at F8 Refresh. Additional information about how to get involved in the beta follows below.

Shared Context

Everyone has a different comfort level with video calls, but breaks or pauses in a conversation over video can sometimes leave people feeling like they have to say something. We think people should be able to hold space together and feel a sense of ease, especially over longer duration calls, much like they would when physically together. AR effects that transport people into what feels like a shared space on their video calls is just one type of experience we’re excited to see creators explore.

Shared Moments

Celebrations big and small may be more virtual now, but they continue to bring people together. We think the Multipeer API can help creators enrich this sense of togetherness with some fun, new possibilities, such as AR effects that use combinations of objects, segmentations, and sounds, all synchronously shared across users and devices.

Shared Games

Finally, we think there’s an opportunity for creators to build effects for group play with lightweight AR games. Whether it’s the simple act of passing an object or working toward a shared goal, the Multipeer API makes it possible for people to play with each other — they can see facial expressions, hear their voices, and feel like they’re together — even when they’re not.

While we’re excited about these new use cases and the possibilities they capture, we’re more eager to see the ideas and experiences that you, our Spark AR community will create. For this reason, we’re now extending our beta program, and inviting all creators to apply to participate. You will need to be familiar with Spark AR Studio and have a comfortable working knowledge of JavaScript.

This is just the beginning of our AR video calling efforts, we’re working hard to create new templates, tutorials and documentation that will help introduce an even larger universe of technical creators to AR creation for video calling. And we can’t wait to see what you’ll create.

Subscribe to the Meta Spark Blog