Spotify using AI to imitate podcast hosts’ voices in different languages, content creators worried

Imagine watching Joe Rogan or any of your favourite podcasters speaking in fluent Hindi. Well, something like that may soon be a possibility. Spotify is in the process of testing a groundbreaking AI feature designed for its podcasts. This innovation aims to translate podcasts into various languages while preserving the distinct voices of the hosts.

In essence, it replicates the host’s voice, intonation, and cadence, maintaining the “speaker’s distinctive speech characteristics.”

Spotify’s $1 billion gamble
While Spotify’s primary focus has been music streaming, the company has made significant strides to establish a strong presence in the podcasting industry, which has seen a remarkable surge in popularity in recent years.

Reports from The Wall Street Journal indicate that Spotify has invested approximately $1 billion to solidify its position in the podcasting realm. This investment stems from the belief that the podcast advertising market, currently valued at $2.3 billion, has the potential to become a more substantial segment within the broader $200 billion digital advertising market.

It’s worth noting that, like the rest of Spotify’s endeavours, podcasting has yet to turn a profit. During an investor day presentation in June 2022, Spotify expressed its expectation that its podcasting venture would become profitable within one to two years.

To test this innovative AI-powered voice translation feature, Spotify has collaborated with a select group of podcasters, including Dax Shepard and Monica Padman, Lex Fridman, Steven Bartlett, and Bill Simmons.

Translated episodes will be made available worldwide on the platform, with Spanish translations set to roll out in the coming weeks. Spotify also has plans to offer French and German translations, although the company has not disclosed a specific timeline for the feature’s broader release. This new feature, rooted in OpenAI’s voice generation technology, truly replicates the podcaster’s voice, resulting in a more genuine listening experience for global audiences. This approach offers a more personal and natural experience compared to conventional dubbing methods, fostering deeper connections between listeners and content creators.

Spotify’s earnings
Ziad Sultan, Spotify’s Vice President of Personalization, emphasized the significance of this innovation by stating, “By matching the creator’s own voice, Voice Translation gives listeners around the world the power to discover and be inspired by new podcasters in a more authentic way than ever before. We believe that a thoughtful approach to AI can help build deeper connections between listeners and creators, a key component of Spotify’s mission to unlock the potential of human creativity.”

Spotify’s financial filings have revealed a notable increase in ad sales from its podcasts, with a growth of €22 million (equivalent to $23.3 million) during the first six months of this year. The company’s second-quarter podcasting ad revenue showed a significant upswing, rising by 30 per cent compared to the previous year.

A bid to salvage their investment
However, the results stemming from Spotify’s deal with Meghan Markle and Prince Harry have been less than stellar. Bill Simmons, who holds a position as a Spotify executive for podcasts, publicly referred to the royal couple as “grifters.” On a podcast episode in June, Simmons recounted an unproductive Zoom call with Prince Harry, which was intended to brainstorm podcast ideas.

During the call, Simmons criticized Prince Harry for his complaints against the British royal family, expressing frustration, and saying, “Shoot this guy to the sun. I’m so tired of this guy.” Simmons questioned the value that Prince Harry brought to the table, particularly highlighting his focus on grievances and public interviews, with a touch of scepticism about the public’s interest in his life.

Simmons remarked, “You live in Montecito, and you just sell documentaries and podcasts, and nobody cares what you have to say about anything unless you talk about the royal family and you just complain about them.”

In addition to these challenges, Spotify has also incurred significant expenses in its efforts to address issues within its podcasting initiatives.

Notably, the company spent $12 million on staff layoffs, and an additional $41 million has been written off for cancelled podcast contracts as part of the strategy to rectify challenges in the podcasting domain.