What is Open AI’s Sora? Text-to-Video Model That Created The Longest AI Music Video

OpenAI’s Sora text-to-video AI model recently, in collaboration with artist ‘Washed Out’, created a music video for the song “The Hardest Part”. This sparked the ongoing debate about the potential of AI as each frame of the music video was generated by it only. This is the longest video yet created by an AI.

Open AI’s Sora holds the potential to revolutionise the Video Production market with its video generation capabilities.

What Is Sora?

Sora is Open AI’s new venture in enabling AI to simulate the motion of the physical world. It is a text-to-video model that can generate minute-long videos based on the user’s prompt. It can generate complex scenes with multiple characters, specific types of motion, adhering to camera movement, and accurate details of the subjects and background.
Sora has an understanding of the physical world and creates relevant scenes instantly based on the user’s prompt.

How Sora Works?

Like Dall-E 3, Sora is another diffusion transformer which means it starts with a static frame of each video. It further uses machine learning to transform the images into the video. Sora videos can be a minute long. However, it has also created a 4-minute-long music video.

What Can Sora Do?

The most prominent use of Sora is that it can create videos on prompts. Moreover, Sora can also convert static images into videos. It can further add special effects to existing videos. Sora also holds the potential to restore the damaged videos. It can remove the subject or background from the existing videos.

Threats From Sora

A video made from Sora is easily recognizable, however, with the speed in advancement of AI, the flaws will become less common. It will be hard to differentiate between AI-generated and real videos. This could be used to create doctored videos aiming to spread disinformation.

The prompt-based video generator could also give rise to existing stereotypes and cultural biases.
Sora is currently not available to the general public. It is in a limited-release phase. It has been made accessible to a select group of artists, filmmakers, and experts for testing and feedback purposes​. The music video is the result of these tests. People are expecting the release in the second half of 2024.