Let's See How Well A Deepfake AI Can Make Cars Lip-Sync

Screenshot: Wombo

I’m not sure how closely you’ve been following the strange new field of AI-enabled deepfake videos and how shockingly accessible they’ve gotten for pretty much any jackass with a computer, but just trust me when I tell you that it’s easier than ever to make plausible-looking videos of people doing things they didn’t actually do. There’s even a goofy/impressive app called Wombo that can make a picture of anyone appear to lip-sync songs. So, it’s time to shove some cars in there.

Advertisement

So far I haven’t seen anyone really attempt to force this Wombo AI—likely a glowing orb in a tank of bluish gel—to anthropomorphize car front ends according to its complex and likely unknowable human face parameters to see how well it works.

As America’s first self-appointed Automotive Anthropomorphization Czar, it’s my responsibility to do so. So I did. And I picked the songs from the limited set available mostly at random, so try not to read too much into that.

I started with one of the most face-like automotive front ends I could think of, an Austin-Healey Bugeye Sprite:

As you can see, it did pretty well! It clearly pegged the headlights as eyes, the grille as a mouth, and the resulting morphed motion definitely feels like what a rubbery Bugeye Sprite would likely look like singing James Brown.

Now, let’s try something more confusing: Luigi, the Fiat 500 from the Cars franchise, which uses the debased windshield-as-eyes conceit instead of the proper, headlights-as-eyes method:

Based on this result, I am not certain the Wombo AI has a preference of its own, as I can’t tell if its reading the windshield irises as eyes or forehead? Either set of eyes is kind of working here. Hm. (The AI has clearly chosen the windshield eyes.—ED)

Advertisement

More study is needed. Let’s try my own Nissan Pao:

I think it made a good call with the bumper-mouth there, and I’m reading the main headlights as eyes, though it’s possible the foglamps confused it? I’m not sure, but I think it works surprisingly well.

Advertisement

What worked surprisingly unwell is a car that I’ve always felt to have a very identifiable face, a classic Volkswagen Beetle:

The lack of a grille/mouth opening seems to have really confused it, making the hood act as a sort of snout. I’m not clear where it thinks the eyes are, as the headlights are all but ignored?

Advertisement

Let’s try something with a much more obvious grille: this International semi truck:

The formatting of the videos here is cutting off a bit at the bottom where a lot of the mouth articulation is happening, but I think this one works pretty well. I believe that truck wants to go to Funkytown.

I picked another obvious mouth-grille here, a Jaguar E-Type, and the result is, yes, the car looks like it’s singing. Even if the AI doesn’t know what to do with the eyes, I think our brains comfortably read the headlamps as eyes in this context, though I think I saw the wipers blink.

Advertisement

Let’s pick a more modern car with a very facial front end, a Mazda 3:

Obvious grilles clearly help this AI succeed. Let’s do one last one, the gloriously worried-face-looking Soviet UAZ 452 van:

Okay, that works, in its way! That old Iron Curtain van clearly is determined to survive. It seems to be singing more out of the side of its grille than the others, but that just adds some character.

Advertisement

I feel like there’s a hell of a lot more important research to be done here, and I strongly encourage everyone to find some good car-faces and try it out, loading the results in the comments so we can all scrutinize and attempt to better ourselves via understanding.

Really, I can’t imagine anything more important any of you would have to do, right?

Senior Editor, Jalopnik • Running: 1973 VW Beetle, 2006 Scion xB, 1990 Nissan Pao, 1991 Yugo GV Plus, 2020 Changli EV • Not-so-running: 1977 Dodge Tioga RV (also, buy my book!: https://rb.gy/udnqhh)

DISCUSSION

I’m done. That’s it. No more Torch for me. I can handle the lunatic obsessiveness over taillights. I can even handle the answers to questions that either nobody thought to ask or were patently obvious from the start. But now we’re into uncomfortably creepy territory, and that’s enough for me.