Meta's Open Source Generative AI Tool AudioCraft Is Here To Create Music And Audio

Meta Platform's AudioCraft consists of three models: MusicGen, AudioGen and EnCodec.

AudioGen was trained on public sound effects and it generates audio from text prompts.

In a bid to create music and audio based on text prompts and audio signals, social networking giant Meta has launched its open-source artificial intelligence (AI) tool called AudioCraft. This is essentially a generative AI model for creating sound and music. Company CEO Mark Zuckerberg made the launch announcement on his Facebook page and wrote: "We're open sourcing the code for AudioCraft, which generates high-quality, realistic audio and music by listening to raw audio signals and text-based prompts."

Meta Platform's AudioCraft consists of three models: MusicGen, AudioGen and EnCodec. MusicGen was trained with company-owned and specifically licensed music and generates music from text prompts. AudioGen, on the other hand, was trained on public sound effects and it generates audio from text prompts.

"Today, we’re excited to release an improved version of our EnCodec decoder, which allows higher quality music generation with fewer artifacts. We’re also releasing our pre-trained AudioGen models, which let you generate environmental sounds and sound effects like a dog barking, cars honking, or footsteps on a wooden floor. And lastly, we’re sharing all of the AudioCraft model weights and code," the company wrote in a blog post.

Meta's AudioCraft family of AI models are capable of producing high-quality audio with long-term consistency. They are said to be easy to use as well.

"With AudioCraft, we simplify the overall design of generative models for audio compared to prior work in the field — giving people the full recipe to play with the existing models that Meta has been developing over the past several years while also empowering them to push the limits and develop their own models," the company added.

Meta is open-sourcing these models and giving researchers and practitioners access so they can train their own models with their own datasets for the first time, and help advance the field of AI-generated audio and music.

AudioCraft works for music, sound, compression, and generation — all in the same place. Because it’s easy to build on and reuse, people who want to build better sound generators, compression algorithms, or music generators can do it all in the same code base and build on top of what others have done.