Nvidia is investing in Mirelo, one of four German startups Jensen Huang announced Monday.
Mirelo's Sound Effect 2.0 generates all sound around a video from an optional prompt, preserves dialogue, and handles ten-minute clips.
Mirelo raised a $41m seed from Andreessen Horowitz and Index Ventures in December and grew from nine to 32 people.
Nvidia backs Mirelo to build frontier audio models for video
Mirelo is building frontier audio models for video. Its Sound Effect 2.0 watches a whole clip and generates all the sound around it at 48 kHz, preserving dialogue and stripping music added by video generators. The bet is that audio needs the same quality of researchers as language and vision models but far fewer GPUs, and that sparse published research lets a small European team build a moat. Nvidia's investment backs that bet.
Takeaways
Nvidia backs Mirelo as a frontier audio bet. Jensen Huang announced at the Invest in Germany summit that Nvidia invested in four German startups, including Mirelo. Mirelo had already worked with Nvidia through its Inception program and contacts with Nvidia researchers before talks with the venture team.
Sound Effect 2.0 generates sound for whole videos. The model takes a video plus an optional text prompt and outputs at full 48 kHz for cinematic quality. It preserves existing dialogue and handles clips up to ten minutes. It also removes music baked in by video generators so users can add their own.
Audio models need far fewer GPUs than language. CJ argues audio needs researchers of the same quality as large language and vision models but at least a factor, possibly two orders of magnitude, fewer GPUs. That lets Mirelo spend a bigger share of capital on researchers, though GPUs still take a large fraction.
Sparse audio research lets Mirelo build a moat. Far fewer publications exist on audio and music models than on language or vision. Mirelo can therefore protect its IP with little risk of being scooped by a paper, while still reusing published improvements such as attention layer changes.
Mirelo grew from nine to 32 people. The $41m seed round, raised with Andreessen Horowitz and Index Ventures, followed a pre-seed of a few million dollars. The new hires worked together on Sound Effect 2.0, the first release from the expanded team.
Europe helps Mirelo recruit frontier researchers. There are not many foundational model companies in Europe, so Mirelo finds it easier to attract research scientists there. It also benefits from research published on large language and vision models and can apply those improvements to its own audio models.
Coherent sound requires stitched cuts, not short clips. CJ argues sound cannot be made coherent at the level of individual ten- or thirty-second cuts, only once cuts are stitched together. Mirelo designed its model for video agents and AI creators, and expects video agents to be a very big part of the market.
What comes next
After Sound Effect 2.0, Mirelo wants to accelerate and double down on the models it builds and what they can do. The Nvidia capital adds people and GPUs for that push.
Tap a name to go there · tap a line to see why they are linked
On this episode 2
Show 1
More resources would also let Mirelo do more side projects. Its open-sourced audio-to-MIDI model transcribes songs and performs far beyond anything published, by a huge margin.