Introducing AR Music for Instagram

Sound is critical to our perception of the world. We think it’s increasingly what separates the good from the great among AR experiences, too. Whether you’re building simple face masks or immersive world effects, Spark AR Studio offers a wide range of sound capabilities to help add depth and dimension to your AR effects, and today we’re adding another with the rollout of music reactive capabilities on Instagram.

Music is an integral part of the Instagram experience, and millions of people use music everyday to add meaning and emotion to the moments they share. Creators can now augment and extend this desire to share by creating AR effects that move and play in sync with music. Here are a few quick examples:

Something Went Wrong

We're having trouble playing this video.

Learn more

Left to right: Themachromic by @positlabs; EQ Glasses by @jerzy.pilch

Left to right: On Kandinsky in Concert by @vi.de.or.bit; Audioled by @malf.visuals; Instabeard by @rosterizer

To create an effect that responds to music, you can either import and use your own legally licensed music files or you can let people pick a song they like from the thousands of free tracks in Instagram’s music library. This feature is now available for Android and iOS users.

The range of what’s possible with audio AR is nearly endless. With this in mind, we wanted to provide some additional tips and considerations for creating effects with music or sound. To get started, import your own sound files or choose from a wide variety of ready-to-use, high quality audio files in the Spark AR Library.

Audio Playback Features

There are several different ways to use audio clips in your project. The simplest way to play a sound is using an Audio Playback Controller, which you can create in the Assets panel and link to a Speaker. To do this, select the Speaker in the Scene panel and choose the audio playback controller from the Audio dropdown in the Inspector. After selecting an imported audio file in the Audio Playback Controller in the Inspector, you can use the Play and Loop checkboxes to control the default playback state for your sound. You can control one-shot and looping playback from the Patch Editor. Audio Playback Controllers can also be used in scripting, via the Audio module.

Advanced Audio Controls

For more complex sound playback applications, you can use the Audio Player and Controller patches. After dragging an audio clip into the Patch Editor from the Assets panel, you can use an Audio Player with either a Single-Clip or Multi-Clip Controller, depending on whether you need to play and loop a single instance of your sound or you want to trigger multiple overlapping instances of the same one-shot.

Microphone Morphing

Voice changer effects are also easy to create by dragging the Microphone from the Scene panel to the Patch Editor, which gives you access to its audio signal output. Audio signal from both the Microphone and the Audio Player can be routed through a variety of audio effect patches, either to change the sound of someone’s voice or to apply live transformations to the audio clips you’re playing in your effect. Any number of audio effect patches can be chained together for more complex audio processing. The resulting output must be connected to the audio input of a Speaker to be audible in your effect.

Tips for Successful Sound Design

Before publishing your effect, make sure to listen carefully to the relative volumes of sounds in your effect and adjust (or “mix”) their volumes so that they fit well together and don’t overwhelm the listener. It’s also important to mix your sounds relative to the voice input to your effect so you don’t drown out the person using your effect if they want to talk while recording.

Another important aspect of sound design is repetitive tolerance — for example, how effective individual sound effects will be through continued use. This can often be improved by lowering the volume of short music loops and repeated one-shot sounds, but you can also alternate between several similar sounds for the same repeated interaction or use audio patches to alter a sound for each playback (like with subtle adjustments to the Pitch Shifter patch).

When designing sound for AR, you can expect that people will hear your work on their mobile phones, which can have distinctive audio playback characteristics. Because of this, we highly recommend testing on multiple different devices, if possible. Mobile phone speakers will generally emphasize the higher frequency range and muffle lower frequencies, and this sort of consideration must be taken into account when creating original sound for your AR effects.

If you’d like to learn more about using audio in Spark AR Studio, check out our audio documentation, including our audio tutorial and sample projects. Additionally, Instagram effects that respond to music are available today with Spark AR Studio (v89), which you can read more about in our blog post.

Subscribe to the Meta Spark Blog