Meta’s researchers have made a significant leap in the AI art generation field with Make-A-Video, the creatively named new technique for — you guessed it — making a video out of nothing but a text prompt. The results are impressive and varied, and all, with no exceptions, slightly creepy.

We’ve seen text-to-video models before — it’s a natural extension of text-to-image models like DALL-E, which output stills from prompts. But while the conceptual jump from still image to moving one is small for a human mind, it’s far from trivial to implement in a machine learning model.

Make-A-Video doesn’t actually change the game that much on the back end — as the researchers note in the paper describing it, “a model that has only seen text describing images is surprisingly effective at generating short videos.”

The AI uses the existing and effective diffusion technique for creating images, which essentially works backwards from pure visual static, “denoising” towards the target prompt. What’s added here is that the model was also given unsupervised training (that is to say, it examined the data itself with no strong guidance from humans) on a bunch of unlabeled video content.

What it knows from the first is how to make a realistic image; what it knows from the second is what sequential frames of a video look like. Amazingly, it is able to put these together very effectively with no particular training on how they should be combined.

“In all aspects, spatial and temporal resolution, faithfulness to text, and quality, Make-A-Video sets the new state-of-the-art in text-to-video generation, as determined by both qualitative and quantitative measures,” write the researchers.

It’s hard not to agree. Previous text-to-video systems used a different approach and the results were unimpressive but promising. Now Make-A-Video blows them out of the water, achieving fidelity in line with images from perhaps 18 months ago in original DALL-E or other past generation systems.

But it must be said: there’s definitely still something off about them. Not that we should expect photorealism or perfectly natural motion, but the results all have a sort of… well, there’s no other word for it: they’re a bit nightmarish, aren’t they?

meta’s make-a-video ai achieves a new, nightmarish state of the art

Image Credits: Meta

meta’s make-a-video ai achieves a new, nightmarish state of the art

Image Credits: Meta

There’s just some awful quality to them that is both dreamlike and terrible. The quality of the motion is strange, as if it’s a stop-motion movie. The corruption and artifacts give each piece a furry, surreal feel, like the objects are leaking. People blend into one another — there’s no understanding of objects’ boundaries or what something should terminate in or contact.

meta’s make-a-video ai achieves a new, nightmarish state of the art

Image Credits: Meta

meta’s make-a-video ai achieves a new, nightmarish state of the art

Image Credits: Meta

I don’t say all this as some kind of AI snob who only wants the best high-definition realistic imagery. I just think it’s fascinating that however realistic these videos are in one sense, they’re all so bizarre and off-putting in others. That they can be generated quickly and arbitrarily is incredible — and it will only get better. But even the best image generators still have that surreal quality that’s hard to put your finger on.

Make-A-Video also allows for transforming still images and other videos into variants or extensions thereof, much like how image generators can also be prompted with images themselves. The results are slightly less disturbing.

This really is a huge step up from what existed before, and the team is to be congratulated. It’s not available to the public just yet, but you can sign up here to get on the list for whatever form of access they decide on later.

TECH NEWS RELATED

Thanksgiving 2022 online sales pip past forecasts at $5.3B, up 2.8% on last year, mobile accounted for 55% of all purchases

Analysts and e-commerce leaders have been predicting a muted online holiday shopping season this year, with sales in the first three weeks of November essentially flat over a year ago due to a weaker economy, inflation, and more people returning to shopping in stores again in the wake of ...

View more: Thanksgiving 2022 online sales pip past forecasts at $5.3B, up 2.8% on last year, mobile accounted for 55% of all purchases

UK to criminalize deepfake porn sharing without consent

Brace for yet another expansion to the UK’s Online Safety Bill: The Ministry of Justice has announced changes to the law which are aimed at protecting victims of revenge porn, pornographic deepfakes and other abuses related to the taking and sharing of intimate imagery without consent — in a ...

View more: UK to criminalize deepfake porn sharing without consent

Karnataka govt proposes 5% commission for app-based auto aggregators Ola, Uber and Rapido

Both Uber and Ola are currently facing the ire of customers in India for random surge pricing. (Image source: Reuters/Representative) Karnataka’s transport secretary on November 25 proposed a commission of 5 percent exclusive of GST to be levied by the app-based auto aggregators Ola, Uber, and Rapido, which is ...

View more: Karnataka govt proposes 5% commission for app-based auto aggregators Ola, Uber and Rapido

Paytm, Zomato top brass rewarded with hundreds of crores of ESOP payouts in H1FY23

(Representative image) New-age companies Paytm and Zomato have recorded share-based payments  worth hundreds of crores each for their top management in the first half of FY23, according to related party transaction disclosures. For Paytm, employee stock option (ESOP) expenses due to key managerial personnel (KMPs), relatives of KMPs and directors was Rs 564 crore ...

View more: Paytm, Zomato top brass rewarded with hundreds of crores of ESOP payouts in H1FY23

5 steps to take your business to the next level

Owning a small business is hard work at any time, let alone right now. In fact, for many owners, the combined impact of the pandemic and the cost of living crisis probably adds up to the most challenging economic circumstances they’ve ever experienced.

View more: 5 steps to take your business to the next level

Leading UK-based Robotics Company BotsAndUs Rebrands as Dexory

The cutting-edge intelligence and robotics company previously known as BotsAndUs will from hereon be known as Dexory it was announced today.

View more: Leading UK-based Robotics Company BotsAndUs Rebrands as Dexory

Elon Musk says Twitter’s new multicoloured verification will launch next week

After messing up the first launch of Twitter’s “power to the people” verification system, Elon Musk said that the social network will tentatively roll out a new multicolored verification system next week. The owner of Twitter said that, under this scheme, companies will get a gold checkmark, government officials ...

View more: Elon Musk says Twitter’s new multicoloured verification will launch next week

Five Ways to Give Back this Black Friday

As millions of shoppers prepare for thousands of deals this Black Friday, Charitable Travel reveals which specialist sites are offering free ways to donate to charity – helping consumers give back at the same time as grabbing a bargain.

View more: Five Ways to Give Back this Black Friday

International digital marketing agency, Eskimoz, expands into UK with the acquisition of Digital Uncut

Taskforce on Climate-related Financial Disclosures a ‘game-changer’ for sustainable lending in the UK, announces Head of Climate at Lloyds Bank

Companies need to become Employers of Choice: Employee engagement and The Great Resignation

A new wave of Solo GP VCs is coming to Europe and Hypernova hopes to power it

South Africa’s Revio allows businesses to connect to multiple payment methods and reduce failures

WeWork China’s former tech head introduces on-demand work pods for mental health

Musk says Twitter will offer “amnesty” to suspended accounts

Twitter layoffs trigger oversight risk warning from Brussels

Stellantis to restructure European dealer network in July 2023

Bessemer, Playground, Root and Seraphim VCs will judge the TC Sessions: Space Pitch-off

Can FTX’s bankruptcy bring order to its chaos?

Automating the income gap

OTHER TECH NEWS

Top Car News Car News