Nvidia Introduces Fugatto, An AI Tool That Can Turn Text Prompts Into Music, Speech, New Sounds

While other tech leaders like OpenAI, Google DeepMind, and Stability AI have entered the AI audio space, Nvidia claims Fugatto’s ability to generate entirely novel sounds sets it apart.

Nvidia has yet to confirm if Fugatto will be released for public use.

Source : Nvidia

Nvidia has launched Fugatto, a cutting-edge AI-powered audio editor designed to create original and unique soundscapes. Billed as a “creative breakthrough,” the tool transforms user prompts—text or audio—into imaginative outputs, producing sounds, music, and speech it hasn't explicitly learned. Examples include a trumpet emitting cat-like meows or a saxophone mimicking barking dogs.

With Fugatto, users can craft audio from innovative and unconventional prompts. One example showcased the tool creating a composition titled “Saxophone howling, barking, and transitioning into electronic music with dogs barking.” The AI can also produce intricate audio layers, such as “low, rumbling bass pulses mixed with sharp, digital chirps, evoking the awakening of a massive sentient machine.”

Fugatto Can Go Beyond Music

Fugatto isn’t limited to music creation. Its capabilities extend to:

• Voice transformation: Modify tones, accents, or emotional delivery, such as shifting a calm voice to sound angry.
• Music editing: Alter compositions by isolating vocals, adding instruments, or replacing elements—for instance, swapping a piano with an opera singer.
• Custom sound effects: Generate effects tailored to detailed textual descriptions.

Developed using a dataset comprising millions of audio samples, including contributions from major sound libraries like the BBC’s archives, Fugatto leverages advanced instruction-based models. Nvidia highlights that this approach allows the AI to learn new tasks without requiring additional training data, enhancing its adaptability and creative potential.

A Step Ahead

While other tech leaders like OpenAI, Google DeepMind, and Stability AI have entered the AI audio space, Nvidia claims Fugatto’s ability to generate entirely novel sounds sets it apart. Most existing tools rely on pre-trained datasets, often producing derivative content, whereas Fugatto pushes the boundaries of originality.

AI In Music: Copyright & Controversies

AI in music creation has sparked debates over copyright, with several startups facing legal challenges. Nvidia, too, has been under scrutiny for its training practices, including the use of YouTube video subtitles in its AI models. Although Fugatto’s development drew from millions of audio samples, Nvidia has not disclosed how licensing and copyright concerns are addressed. However, its focus on producing wholly unique outputs might help it navigate potential legal challenges.

Despite its game-changing potential for artists, filmmakers, and sound designers, Nvidia has yet to confirm if Fugatto will be released for public use. For now, the tool remains a fascinating glimpse into the future of AI-driven audio innovation.