How Badly Will Deepfakes Weaponize Generative AI?

Has deepfaking hijacked generative AI for good? Deepfake AI is a type of artificial intelligence used to create convincing images, audio and video hoaxes. The use of deepfaking to deceive people has always been a concern. But recent developments in the development of deepfakes through generative AI have elevated that concern to a DEFCON 1 emergency. Businesses that develop generative AI are under increased pressure to develop the AI responsibly.

Why Is Deepfaking Becoming a Bigger Problem?

Deepfakes are created using a combination of techniques, including face swapping. Deepfakes are created using a type of AI called deep learning. Deep learning algorithms are trained on large datasets of images or videos of a particular person. The algorithms learn to identify the unique features of that person’s face, such as the shape of their nose, the size of their eyes, and the position of their eyebrows.

Once the algorithms have learned these features, they can be used to create a new image or video in which the person’s likeness has been replaced with someone else's. The new image or video will be very realistic, and it may be difficult to tell that it has been manipulated. The results can be quite convincing.

Deepfakes are often used to create fake news, misleading videos, and pornography without the consent of the user. The use of deepfakes is raising concerns about the potential for abuse. Those concerns have intensified with the proliferation of generative AI tools that make it easier for anyone with access to commercially available software to commit very convincing deepfakes. By making generative AI more accessible, businesses are unwittingly contributing to the potential spread of harmful content.

Adobe Opens the Floodgates

For instance, Adobe recently announced the launch of a version of Adobe Photoshop that makes it easier for anyone, regardless of their technical proficiency, to alter an image in sophisticated and sometimes disturbing ways without mastering complex software. Previously, using Photoshop required a certain level of graphics expertise to, say, remove an unwanted object such as an unsightly garbage can or power line wires from a photo. But with the latest version of Adobe, anyone with access to the software can easily manipulate images in ways never before possible. As The Washington Post reported, Photoshop can be used to completely alter the meaning of a photo, including adding or removing people, changing a city’s skyline, and so on.

The Generative AI Problem

Adobe embedded Photoshop with generative AI as a response to the explosion of powerful generative AI tools such as DALL-E 2, Midjourney, and Stable Diffusion. As these tools continue to add features and levels of sophisticated in an attempt to outdo each other, they’re also increasing the risk of deepfaking with potentially harmful results. For example:

• A Midjourney user, in an attempt to demonstrate how the software can be used for deepfaking, created fake images of politicians cheating on their spouses. He was promptly banned from Midjourney "I got banned from Midjourney AI for generating realistic images of politicians cheating on their wives for a series called 'AI will revolutionize the blackmail industry,'" the images’ creator, video editor Justin Brown, wrote in a Twitter thread showcasing the images. He adding “when used with intelligence and intent, AI can be weapon.” Midjourney banished Brown after he posted the images on Reddit -- which suggests that others could be using Midjourney to making disturbing deepfakes in stealth mode, and of course bad actors will not post their images on Reddit to make a point.
• Recently a phony image of an explosion at the Pentagon, suspected to be a deepfake, went viral. The Dow Jones Industrial Index dropped 85 points within four minutes. Making matters worse, the phony image was posted on a fake Bloomberg News account that had been verified under Twitter’s paid verification system – adding to the illusion that the misinformation had come from a legitimate news source. The image was called out and removed, but not without causing a ripple in the stock exchange not to mention any stress incurred by families of people working in the Pentagon, which, as we know, was attacked for real on 9/11.

Deepfakes are already being used to blackmail people by creating phony images of them engaged in pornographic situations. This form of blackmailing is known as sextortion. This criminal activity has been happening for the past few years, and it is going to get worse and more damaging to society at large, examples being:

• Widespread manipulation of financial markets.
• Even more malicious criminal attacks on businesses and people through organized crime.
• Massive deception of voters and ultimately an attack on Democracy itself, especially in the run-up to the 2024 elections in the United States.

What Is the Solution?

What is the solution? Well, businesses that develop generative AI are aware of the problem, and they do have safeguards in place to try to contain deepfakes. For instance, Adobe told the Washington Post that it is also building some limits into its AI products. Once the AI-powered version of Photoshop comes out of beta, it will automatically include so-called content credentials in the files it produces that would flag if an image has been altered with AI. But the Washington Post noted that that safeguard was not in place with the version its reporters tested. According to WaPo:

Adobe says the images its AI produces are “pre-processed for content that violates our terms of service,” such as pornographic content. When it spots violating content, it blocks the prompt from being used again.

We found this to be hit or miss: Sometimes it was overly sensitive — like stopping us when we asked to add a UFO to an image. Other times it seemed not sensitive enough — like when we asked to add the face of a baby to an infamous photo of Kim Kardashian that ‘broke the internet.’

WaPo also noted that Adobe is participating in an industry effort called the Content Authenticity Initiative, “but the jury is out on whether that will be enough.”

For its part, Midjourney shut off free access to its free trial version earlier in 2023 due to the proliferation of deepfakes using the software. Clearly, we are seeing a consequence of Silicon Valley’s “release a version to the public and make the product better” approach as we did with OpenAI. By making the software widely available, Midjourney called attention to its vulnerabilities, which, it could be argued, was a positive step toward making the AI more responsible. But simply taking the toys away from malicious children isn’t the solution. The businesses that create the software have a responsibility to develop it with proper safeguards in place. Since they developed the software, it’s on them to find solutions. If they don’t, legislators will do so for them. 

Currently, Midjourney tries to ferret out deepfakes through a number of measures, such as banning the use of certain words to create images and reliance on human content moderation. But these measures won’t work without humans being in the loop to:

• Train the data used by AI, flag problems that arise (such as inappropriate content creeping into images) and course-correct the app.
• Moderate content. Content moderation is never easy, but machines cannot do it alone. Machines allow too many abuses to occur and too often flag legitimate uses of AI. Human judgement is needed.
• Manage security and copyright, including unauthorized use of the app.

Human beings created Midjourney and its competitors. Human beings need to manage what we created.