Throughout the past year, generative-AI technology has significantly advanced in the quality and quantity of outputs, with major expansions in their availability and proliferation of use. While these advancements reflect significant achievements for innovation and the technology industry at large, they have prompted major concerns for protection of individuals’ rights. These AI technological advancements have been utilised to develop deepfakes – AI-generated visuals and audio outputs that mimic real people’s identities, synthetically cloning individuals’ looks, voices, and mannerisms in a way consistent with how the individuals authentically present themselves in real life.
Estimates have been made of a 900% annual increase of the amount of deepfakes generated since 2023. With resolution now high enough, deepfakes have become a potential mechanism for fraud, harassment, blackmail generated with the intention to deceive. These advancements enable their outputs to be bypassed by AI detection systems, limiting the capacity for take-down mechanisms.
Among the various AI tools of concern is Grok AI – a generative AI chat bot developed by Elon Musk’s company x.AI. The tool is integrated into the social media platform X, lending itself to easy and thus widespread use. Users can input a prompt into the chat bot, with Grok AI producing a highly realistic audio-visual output instantaneously. The tool has been utilised to generate pornographic or sexually explicit deepfakes without the consent of the identifiable individuals its outputs depict. Analysis published by AI Forensics determined that 53% of images generated by Grok AI contained visualisations of individuals with minimal attire (81% of which presented women) and 2% contained visualisations of individuals presenting to be 18 years or younger.
Test case in the UK
On 3 June 2026 Labour MP for Lowestoft, Jess Asato, filed a claim in the High Court for misuse of private information and breach of data protection against x.AI, after a Grok tool had assisted a user in producing fake sexualised images of her in a bikini, and video relating to sexual assault. Jess Asato is claiming as part of her relief: damages, a declaration of illegality by xAI, and an order prohibiting xAI from carrying out further illegal conduct. Upon issuing her claim, Jess Asato stated: “My hope is that this will rebalance individuals’ rights against very large tech companies that should have put safeguards in place before they harmed women and children.”
Regulatory Response
Following the European Commission launching an investigation on the matter late January, on 3 February 2026, the UK’s independent regulator for data protection – the Information Commissioner’s Office (‘ICO’) – announced its formal investigation into X and x.AI after numerous complaints that Grok was being used to generate sexual images of real women and children. The inquiry essentially seeks to determine whether X and x.AI have been compliant with UK data protection law. This entails determining whether the design and deployment of the Grok AI system has appropriate safeguards in place to prevent the system from using an individual’s personal data as inputs to generate explicit deepfake outputs without the subject’s consent.
Consequently, potential exists for individuals to have had their data protection rights infringed. The sexualisation of one’s personal data without consent constitutes an individual’s loss of control over their personal data, which can expose them to immediate and serious harm.
In a statement given on 3 February 2026, William Malcolm, Executive Director Regulatory Risk and Innovation at the ICO, said:
“The reports about Grok raise deeply troubling questions about how people’s personal data has been used to generate intimate or sexualised images without their knowledge or consent, and whether the necessary safeguards were put in place to prevent this. Losing control of personal data in this way can cause immediate and significant harm. This is particularly the case where children are involved.”
The outcome of the ICO’s investigation remains to be seen, but, if it finds X and x.AI have failed to meet their data protection obligations, it has a range of enforcement powers. These include the issuing of fines of up to £17.5 million or 4% of the organisation’s annual worldwide turnover – whichever is higher. Ofcom, the UK’s communication services regulator, is also investigating X under the Online Safety Act, having launched a formal investigation on 12 January 2026. In serious cases of ongoing non-compliance, Ofcom has the power to apply to the court to block access to a site in the UK, which would be a significant regulatory invention given the impact on freedom of expression and potential political implications.
For more information on Deepfakes, please see here and here.