17/03/2026
Introduction
Over the past decade, democracies have been increasingly targeted by information manipulation and interference (FIMI) operations conducted at the behest of state actors. The tools of manipulation and interference have evolved beyond fake news headlines and bot-generated memes. Artificial Intelligence (AI) is now expanding the toolkit, and so-called deepfakes are the current craze. Written from a legal perspective, this blog post discusses the capacity of the recently introduced EU AI Act to address them.
Deepfakes: what they are and why they matter
Deepfakes are a subset of AI-generated synthetic media created using a type of advanced machine learning techniques, deep neural networks or ‘deep learning’, specifically, generative adversarial networks or GANS. GANS are a class of deep machine learning consisting of two neural networks, a generator that creates synthetic content and a discriminator that evaluates its realism, trained together in a competitive process (Zheng et al., 2025). GANS are used to fabricate audiovisual content. By learning and mimicking the patterns of real human faces, voices, or movements, synthetic media are difficult to distinguish from authentic content.
In the context of FIMI, deepfakes raise particular concern due to their realism and impact. They can impersonate public figures, manufacture events, or simulate statements that were never made, all in a persuasive format. Unlike traditional forms of propaganda, deepfakes benefit from audiovisual credibility, speed of production, and viral potential. Elusive, difficult to trace, easy to reproduce, deepfakes are especially dangerous in a media landscape where public opinion can swing based on a single clip. For example, in January 2024 an AI-generated robocall impersonating former US President Mr. Joe Biden urged Democrats not to vote in the New Hampshire primary (Steck & Kaczynski, 2024). This deepfake was meant to mislead voters and undermine elections. In another case, in February 2025, social media users shared a video showing the US President Mr. Donald Trump speaking about a phone call with the Russian President Mr. Vladimir Putin. This video was later found to be a ‘lip-sync deepfake’ from an earlier press briefing video of the US leader (Cheng, 2025). It has been observed that the use of deepfakes impersonating political figures and/or spreading disinformation, and distorting public debate increases during election periods (Jacobsen & Simpson, 2023).
The regulation of deepfakes under the EU Artificial Intelligence Act (AI Act): the Obligations to Mark and Disclose Synthetic Content
The AI Act, adopted in 2024, is the world’s first legal framework on AI. Its scope covers any AI system, including fixed-purpose AI applications (also known as narrow AI) and general-purpose AI systems, i.e., AI systems that have a wide range of possible uses, both intended and unintended by the developers (Article 1 and Article 1b). The Regulation adopts a risk-based approach (Recital 26), which means the higher the risk of AI systems causing harm, the stricter the rules. AI systems posing an unacceptable risk are downright prohibited (Article 5), such as AI systems for the evaluation or classification of persons based on their social behaviour (social scoring) or an AI system that deploys subliminal techniques beyond a person’s consciousness or purposefully manipulative or deceptive techniques.
High-risk systems referred to in Article 6(2) and Annex III are AI systems used in sensitive areas such as employment and occupation, education, law enforcement, etc. They are allowed insofar as they comply with a series of obligatory measures, on transparency, human oversight, literacy measures, risk management, due diligence, conformity assessment, etc. (Articles 8 to 49). The Regulation further distinguishes ‘certain’ AI systems that interact with natural persons, and which pose specific transparency risks (see Recital 132), such as chat boxes and, indeed, deepfakes (Article 50).
Specifically, the AI Act explicitly mentions deepfakes in Article 3(60), which defines them; in Article 50, which sets out the transparency obligations for certain AI systems; and in Recital 134, which alludes to the tension between deepfakes and freedom of expression. Article 3(60) states: ‘Deepfake means AI-generated or manipulated image, audio or video content that resembles existing persons, objects, places, entities or events and would falsely appear to a person to be authentic or truthful’. The definition implicitly introduces an important distinction between deepfake makers or generators, i.e., the AI systems, and deepfakes as the output of those systems. While Article 3(60) defines deepfakes as output, as this blog post shows, the obligations of the AI Act apply both to (providers of ) AI systems generating deepfakes and to (deployers of) deepfakes as output.
Under the AI Act’s categorisation, AI-systems generating deepfakes are arguably classifiable as general-purpose AI systems, given that they enable the flexible generation of content, such as text, audio, images, or video, and accommodate a wide range of distinct purposes and tasks (Recital 99 and Article 3(63)). Furthermore, under the AI Act’s risk categorisation, due to this flexibility in use and purpose, deepfakes may be prohibited, classified as high-risk, or fall outside those categories, depending on how they are used, for what purpose, and to the extent they impact on individual rights and freedoms.
Accordingly, AI-systems generating deepfakes are to be considered unacceptable and thus prohibited in case they generate deepfakes with the intent to manipulate individual or group behaviour, for example, when deepfakes are shaped and circulated as news articles (Article 5(a), Recital 29 of the AI Act). Alternatively, AI-systems generating deepfakes may be classified as high-risk if they are deployed in one of the areas mentioned in Article 6(2) and listed in Annex III. For instance, educators and museum curators already use deepfakes to make history lessons more interactive and accessible, or to recreate historical figures and animate archival footage (Krishna, 2020). In such cases, the task deepfakes perform does not pose any significant risk. However, if the use of deepfakes were to impinge on ‘educational assessment’ (Annex III, AI Act), for example, if they were surreptitiously used to evaluate learning outcomes, steer the learning process of natural persons, or monitor and detect prohibited behaviour by students during tests, such systems would be considered high-risk and must therefore comply with the stricter obligations of the Regulation.
As mentioned above, the Regulation recognises that deepfakes may also form part of an ‘evidently creative, satirical, artistic, fictional, or analogous work or program’ (Recital 134), which does not cause any harm, no malicious deception, or manipulation. This typically occurs in contexts such as theatres, cinema halls, or art exhibitions, where viewers do not expect to receive truthful information and can easily recognise the content as fictional. In these scenarios, deepfakes (systems and outputs) do not infringe upon individual rights
and are therefore not considered high-risk. This would be the case, for example, of the film The Brutalist where an AI software system for voice cloning and manipulation was deployed to enhance the Hungarian pronunciation of its two lead actors, altering their voices.
Given their multiple purposes and bearing in mind the distinction between deepfake systems and outputs, the key provision on the regulation of this technology can be found in Article 50 of the AI Act.
Article 50 sets out specific transparency obligations addressed to providers, i.e., developers (Article 3(3)) of AI-systems generating deepfakes, and to deployers, i.e., users (Article 3(4)) of AI-generated or manipulated image, audio or video content. These transparency obligations apply to any AI system, whether general-purpose, narrow or specific, as long as ‘intended to interact with natural persons or generate content.’ Even if not categorised as high-risk or originally designed or intended for deceptive purposes, these systems may still pose risks of impersonation or manipulation (Recital 132). Therefore, a basic level of transparency is required to ensure that people are aware they may be dealing with synthetic content.
Article 50(2) stipulates that providers of AI systems ‘generating synthetic audio, image, video, or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated’. Although they are not explicitly mentioned in Article 50(2), the expression ‘generating synthetic audio, image, video, or text content’ includes deepfakes, given that deepfakes are synthetic media. The obligation for providers is to ensure that the outputs of the AI system are marked in a machine-readable format and are detectable as artificially generated or manipulated. This applies to any kind of media, including text.
Article 50(4) states that deployers of an AI system that generates or manipulates image, audio, or video content constituting a deepfake, ‘shall disclose that the content has been artificially generated or manipulated’. Interestingly, the wording of Article 50(4) does not, as paragraph 2 of the same article does for providers, explicitly mention text-based deepfakes, arguably because the provider’s technical solutions may be sufficient to achieve the intended transparency purpose. Instead, artificially generated or manipulated text is addressed only in a limited context: namely, when such content is published to inform the public on matters of public interest. This stands in contrast to Article 50(2), which imposes, as already mentioned, a broader obligation on providers to mark all types of synthetic content, including text, in a machine-readable and detectable format, regardless of context.
This difference in how the rules apply isn’t accidental; instead it reflects the AI Act’s careful attempt to regulate deepfakes at different stages of their lifecycle. It is crucial to recall the earlier distinction between deepfakes as output and the AI systems that generate them. The obligation imposed on developers, namely to mark synthetic content, regulates the process of generation itself. By contrast, the deployer’s obligation to disclose arises when deepfake content is released or made accessible to the public, i.e., when it is likely to be seen by or interact with natural persons..
At the same time, Article 50 introduces some exemptions to the marking and disclosure obligations. It specifies that the provider’s marking obligation does not apply where AI systems perform an assistive function for standard editing or do not substantially alter the input data provided by the deployer, nor its semantics. Additionally, transparency obligations, whether marking or disclosure, do not apply when AI-generated synthetic media is used for law enforcement purposes, for example, when law enforcement agencies (LEAs) are investigating crimes or gathering evidence. A question for another time, perhaps, would be about deepfakes produced by LEAs for entrapment purposes, for instance in child pornography. Would this be considered part of ‘investigating crimes’ under the new regulation? For journalism, the AI Act stipulates that AI-generated or manipulated text published to inform the public about topics of public interest must generally be disclosed, except where the text has been reviewed by a human or overseen by an editor who assumes responsibility for its content (Article 50(4), Recital 134). Finally, as already mentioned, disclosure obligations do not apply if the content is part of an ‘evidently’ artistic, satirical, or fictional work. The insertion of ‘evidently’ is meant to ensure that labels or signposts do not interfere with the work itself, thereby preserving its value and quality and safeguarding the rights to freedom of expression and freedom of the arts and sciences (Recital 134).
Lastly, the Regulation provides some guidance on how the obligations to mark and to disclose can be implemented. This can take the form of labels, signposts, watermarks, or other technical indicators designed to signal that the content is not authentic (Recital 133 of the AI Act). However, as noted in the scientific literature, watermarking and content provenance mechanisms are not wholly devoid of risks (Fragale & Grilli, 2024). Guidelines or standards may therefore be necessary for techniques marking deepfakes to ensure greater legal and technical certainty (Meding & Sorge, 2025). The Regulation offers general guidance concerning the clarity of communication, stating that the information that the content is not authentic or AI-generated shall be conveyed ‘in a clear and distinguishable manner at the latest at the time of the first interaction or exposure’.
Can the EU AI Act Effectively Address Deepfake Risks?
Deepfakes are no longer just digital curiosities. They can amplify disinformation campaigns and undermine democratic processes, such as elections. The AI Act promotes a culture of transparency around the design and use of these technologies, which is aimed at informing individuals and holding developers and deployers accountable for the design and use of these potentially dangerous media. It does so by introducing, in Article 50, specific transparency obligations. Developers are under the obligation to ensure that AI systems generating deepfakes mark the content as AI-generated. Deployers are under the obligation to disclose, i.e., to release information that the content has been generated by AI. The transparency obligations are welcome, but will they suffice?
First, while the AI Act does offer a working definition of ‘deepfake,’ it does not, and probably, cannot specify the threshold or set criteria for singling out misleading, or deceiving content as illegal. This legal uncertainty highlights a deeper tension, that Recital 134 recognises, between regulating deepfakes and safeguarding freedom of expression.
Second, Article 50 displays some inconsistencies. Deepfakes are explicitly mentioned under deployers’ obligation to disclose (Article 50(4)) but not under the providers’
obligation to mark, although they are arguably included as synthetic media (Article 50(2)). Text-based synthetic content is mentioned under providers’ duties but not at the deployment stage. These apparently small inconsistencies may create loopholes especially concerning text-based deepfakes like fake news articles or AI-generated messages (Vanberghen, 2024; Meding & Sorge, 2025).
Third, guidelines may be required to implement transparency measures. As mentioned, watermarks signposts and content provenance tools themselves are not without risks (Fragale & Grilli, 2024). Without common technical standards or guidelines, there is a risk of fragmentation across the EU. Thus, the effectiveness of enforcing the obligations to mark and disclose remains to be seen. Differences in capability, expertise, and political will among national supervisory bodies could weaken consistent application, allowing deepfake developers to exploit differences across jurisdictions.
Furthermore, the spread of deepfakes through social media may pose additional enforcement challenges. While the regulation imposes obligations on traditional media, where a human or editor assumes responsibility for determining whether to disclose that content has been AI-generated, such oversight mechanisms are not always present on social platforms. Although the AI Act provides a promising foundation for regulating deepfakes, its effectiveness will arguably depend on how well it aligns with the obligations of online platforms under the EU’s Digital Services Act (DSA). More on this in the next blog post.
Written by: Afroditi Papathanasopoulou and Eugenio Mantovani (VUB)
References:
Krishna, Dhruva (2020). “Deepfakes, online platforms, and a novel proposal for transparency, collaboration, and education.” Rich. JL & Tech. 27: 1.