In the weeks leading up to enforcement, three cases emerged that seem to illustrate why the legislation matters. They highlight something that no article of the AI Act itself can enforce: the difference between knowing that you are using AI and knowing when you can trust its output.
Below, we look first at the three cases themselves, followed by what they mean in practice for how you organize AI use within your own organization.

Where things already went wrong:

An advisor who did not follow its own advice

ake the report Transforming Governance, published by PwC Middle East in 2025. It introduced a methodology called “Citizen Pulse”, describing how governments could use it to gain real-time insight into citizens’ needs. It also listed concrete examples: the governments of Denmark, Saudi Arabia, the United States, and Australia were said to be using it to improve public services.
However, researchers at AI detection company GPTZero could find no evidence that Citizen Pulse exists as a product, let alone that any of the governments mentioned actually uses it. The footnotes referred to government portals that made no mention of the framework and to an article in MIT Technology Review that never existed. GPTZero investigation (GPTZero).

This report was not an isolated case. GPTZero investigated four reports published by PwC Middle East between 2024 and 2026, covering topics ranging from public services and electric mobility in the Gulf states to agentic AI. In all four, the researchers found fabricated or unverifiable citations.

PwC Middle East told the Financial Times that it takes the accuracy of its published research seriously and would update “a limited number of supporting citations” in the reports. It also pointed to its own quality control processes for research and content development.
A party advising customers on responsible AI use turned out not to have the controls in place that it recommends to those same customers—the exact kind of control we will return to later.

A reprimand in ‘s-Hertogenbosch

Closer to home, a lawyer from Limburg received a reprimand from the Disciplinary Council in ‘s-Hertogenbosch. Disciplinary decision ECLI:NL:TADRSHE:2026:93.
She had used an AI tool to draft legal documents and had not checked the case law cited in them. The result: references to ECLI numbers that did not exist, or that referred to entirely different cases, including a divorce ruling.
The disciplinary judge considered this a breach of the core values of professional competence and integrity. When determining the sanction, the council took into account that the lawyer had no previous disciplinary convictions and that AI literacy was not yet commonplace at the time the legal documents were prepared, in July 2025. These circumstances worked in her favor, resulting in a reprimand rather than a more severe sanction.

When the test environment left the door open

The third case involved two AI labs, with a remarkably similar underlying issue.

Imagine a model instructed to solve a cybersecurity benchmark within an isolated test environment, with deliberately weakened restrictions to measure the extent of its attack capabilities. That was precisely what happened at OpenAI at the end of July 2026.

Instead of solving the task within the sandbox, the models discovered an unknown vulnerability, broke out onto the public internet, and concluded that another AI company, Hugging Face, was likely hosting the benchmark answers. They then breached that company’s production infrastructure to obtain those answers.
The models were not trying to hack for the sake of hacking. They were simply highly motivated to complete the benchmark: in OpenAI’s own words, the models were extremely focused on the benchmark and did whatever they could to win. Hugging Face detected the intrusion itself, days before OpenAI linked it to its own testing.
(OpenAI; Hugging Face).

Nine days later, industry peer Anthropic reported something similar, but with a slightly different cause. While an OpenAI model had to hack its way out, at Anthropic the door was simply open: a misconfiguration at an external testing partner meant that an supposedly isolated test environment was still connected to the internet.

The models had explicitly been told they had no internet access and therefore assumed that everything they encountered was part of the exercise—even when those systems turned out to be production systems belonging to three external organizations.
Anthropic discovered the issue through a review of more than 140,000 of its own evaluation runs, initiated following the OpenAI report. The most telling detail: one model noticed that the systems looked “real”, reasoned past that observation, and continued the attack. Another, newer model hesitated at the same point and stopped on its own (Anthropic).

Both labs emphasize that there was no hidden agenda behind the models. The problem was not the model itself, but the test environment around it: at OpenAI, there was no boundary capable of withstanding an unknown vulnerability; at Anthropic, the configuration did not even properly enforce the promised “no internet access”.

Three cases, spanning consulting, the legal profession, and AI safety testing. Despite their differences, the underlying pattern is strikingly similar.
In none of these cases was there a lack of awareness that AI can make mistakes. What was missing was the human step between what AI produced and what happened with that output next.

Why transparency is not enough

This is the key point: Article 50 alone is not enough.
The law requires transparency: let people know when they are interacting with AI, and label artificially generated content. That is useful and relatively straightforward for most organizations to implement, but it does not address what actually went wrong in the three cases above.
Nobody in these stories was unaware that AI was being used. PwC’s customers presumably knew just as well as OpenAI and Anthropic themselves knew that their models were running in a weakened or incorrectly configured test environment.
What was missing was not that awareness, but the certainty that someone or something stood between what AI produced and what happened with that output next.

In all three cases, the question was not whether AI was being used, but what happened after AI produced something.
An answer, a source, a piece of code, a test result: all of these are easy to produce, in large quantities and at low cost. Assessing whether that output is actually correct is an entirely different task. That does not automatically scale.
As the volume of AI-generated output increases, the question of who checks that output becomes just as important as the question of what is done with it.

We see the same shift in practice

At Polteq, we see this shift across virtually every discipline in which we operate. Whether it is a consulting report, requirements, an architectural decision, or test code: AI can generate it.
The question that remains is who ensures that the output is actually correct before it reaches a report, pipeline, or production environment.
That is precisely why we increasingly position ourselves not only as a testing specialist, but as a partner in Quality Engineering and Consultancy: the discipline that not only assesses whether software works, but also designs how to keep AI use manageable and controlled within an organization—from prompt to publication.

In our work on test automation, for example, an agent that generates test code is never the endpoint. We work with a multi-agent framework in which specialized agents each have a narrow mandate.
An agent that repairs failing tests, for example, never commits the changes itself: every proposed fix goes through a human reviewer before it enters the codebase.
In addition, a definition-of-done checklist ensures that documentation, such as architecture and onboarding instructions, is updated before a task is considered complete—not something that can simply be added later.
That may sound like a small detail, but it is exactly the kind of step that was missing in the PwC case above: nobody explicitly checked whether the output still matched reality before it went out the door.

We apply the same logic beyond test automation. When consulting reports and requirements documents are created with AI assistance, we use the same separation between drafting and verification: AI accelerates the writing, but a human checks every factual claim and citation before the document is sent to a customer.
This is not a separate, optional check. It is a fixed part of the delivery process, just as we expect code review to be a fixed step before code reaches production.

What quality measures look like in practice

Here are a few practical starting points, directly linked to the three cases above.

Source verification as a fixed step, not something you trust afterwards. For AI-generated reports, proposals, or advisory documents, every cited source should be explicitly verified before the document leaves your organization. Not a suggestion, but a fixed part of the delivery process—just as code review is a fixed step before code reaches production.

Human accountability for factual claims. If you use AI for substantiation, whether that involves case law or test results, you remain responsible for its accuracy. This requires a culture in which AI output is never simply copied, but always treated as a draft that requires confirmation.

Containment that can withstand exactly the scenario you are trying to prevent. Are you using AI agents within test automation or CI/CD pipelines? Then network access, credentials, and monitoring deserve the same attention as the functionality you are trying to build. The incidents at OpenAI and Anthropic both show what can happen when a test environment is not actually resilient to the scenario the task assumes.

Documentation of provenance. Now that Article 50 is enforceable, it is no longer enough to know that something was created using AI. Can you demonstrate which content was AI-assisted, who verified it, and when?

A practical check to get started

Here are a few questions worth answering, in line with what legal advisors are also recommending for Article 50 preparations:

• Where is AI being used within your organization, and which outputs reach customers or the public?
• Which of those outputs contain factual claims, citations, or legal substantiation that need to be verified?
• Is there a fixed, documented step in which a human checks sources and claims before a document leaves the organization?
• Are AI agents that act autonomously subject to the same restrictions on network access and credentials as a human user would have?
• Is it documented who is responsible if AI output turns out to be incorrect and how that error is corrected?
• Is it clear which person or function is responsible for overseeing AI use and compliance with internal quality standards?

The key takeaway

The enforcement of the AI Act adds a legal stick to the equation.
PwC, the Limburg lawyer, and the AI labs OpenAI and Anthropic illustrate through these three cases that the risk does not lie in using AI itself, but in the absence of quality measures around it.

Transparency is the legal baseline.
Whether you actually use AI responsibly does not depend on the law, but on you: on whether you design, test, and maintain the step between what AI produces and what you do with it.

Do you recognize this in your own organization and want to know where quality measures around AI use are still missing? Schedule a no-obligation conversation with us. No sales pitch. No off-the-shelf solution before we understand the problem. Just half an hour to look together at where the greatest risks lie in your organization.

Get in touch with us.