The hidden costs of AI
A significant portion of the tokens we consume every day is spent on polite detours, unnecessary confirmations, and overqualified language models handling simple tasks.
Because I regularly reflect on resource consumption, environmental impact, and sustainability, I decided to take a closer look at the underlying flow of my own AI workflows. It is worth realizing how much computing power quietly disappears into the peripheral aspects of a conversation. The urge to immediately reach for the largest and most versatile cloud model is understandable. We expect AI to seamlessly complete our thoughts, solve complex problems in one go, and remain pleasant to interact with at the same time. But the accumulation of those expectations has created a blind spot.
The costs add up remarkably fast. Researchers at the University of California, Riverside calculated for The Washington Post (September 2024) that generating a simple 100-word email with GPT-4 consumes approximately 0.14 kWh of electricity—roughly the same as running fourteen LED bulbs for an hour—plus 519 milliliters of water for cooling. That energy is not simply spent on generating text. Behind the scenes, keeping massive GPU clusters running in data centers requires substantial amounts of electricity and cooling water. For every word generated, billions of model parameters must be recalculated. Of course, politeness remains important in human communication, but an AI system does not care whether a “thank you” is sincere or merely habitual: every extra interaction still activates the servers. In my own daily usage, I quickly grew tired of all the performative politeness and lengthy unsolicited explanations AI tends to provide at every step. Together, they consume a surprising number of output tokens—and therefore compute time—without adding much value.
The hidden cost of language
Another, more technical insight was the impact of language choice itself. I noticed that I frequently mixed English and Dutch within my prompts. That feels perfectly natural to me, but it has a direct effect on token usage.
Most of the language models in my toolkit are trained and optimized primarily on English-language data. Their tokenizer—the system that breaks text into small, manageable pieces for the neural network—is therefore often more efficient in English. Depending on the model and tokenizer being used, English can sometimes express the same idea with fewer tokens than Dutch. When communicating purely functionally with an agent, English is often “cheaper” to process.
Choosing a language therefore becomes a trade-off. For functional code and scripting tasks, I tend to use English to reduce token consumption. But when nuance matters, I prefer my native language, Dutch, because it allows the message to land exactly as intended.
Building a more efficient toolkit
With this in mind, I started redesigning my working environment. Wherever possible, I now separate reasoning from execution.
My general rule is simple: complex, nuanced writing and thinking tasks go to a larger model such as Sonnet or Opus, while preparatory sorting, pattern recognition, and lightweight data analysis are delegated to a smaller, much faster Haiku model. Only when quality noticeably drops do I scale up to maximum reasoning modes or more advanced models such as Fable.
For operational work, I also use my own “caveman mode,” based on an open-source prompt concept that I adapted for personal use. There are several levels. The mildest version removes unnecessary apologies and lengthy explanations at every reasoning step. The strongest version, however, is also the most entertaining.
There is something wonderfully ironic about forcing an advanced state-of-the-art language model to communicate in primitive shorthand. Responses such as “TEST PASS. BUILD GOOD.” are genuinely amusing to watch scroll across the screen. Omitting prepositions and reducing grammar to its bare essentials may sound blunt, but it significantly cuts the number of tokens generated per interaction. In my own experiments, I observed little to no loss in quality. What disappeared was mostly surrounding explanation, not the quality of the work itself.
The move toward local models
Beyond routing tasks intelligently in the cloud, I have also been experimenting with local models.
One of the open-source projects I am currently exploring allows a large model to run primarily on consumer hardware by loading only the few specialized subnetworks—or “experts”—required to predict the next word. The rest remains stored on disk until needed. That principle—activating only what you need—is, at its core, what efficient AI usage is all about.
It still consumes energy on your own hardware, of course, but it demonstrates how much is now possible on a machine you fully control.
This introduces some interesting trade-offs. On average consumer hardware, local generation often sacrifices a considerable amount of speed. In some cases, generating even one or two tokens per second can be a challenge. Yet the benefits are significant. Your data never leaves your own network, there are no variable costs per generated token, and you remain independent of an internet connection.
More importantly, local models force you to think critically about scale. Do you really need a trillion-parameter model for a simple test analysis, or would a small, focused local model already be sufficient? Some tasks can also afford to take longer. Just as a large test suite can run overnight in the background, a local model handling bulk work does not always need to provide an immediate response.
Innovating through curiosity
Using AI infrastructure more efficiently is not merely a way to keep cloud services affordable.
Over the past year, Mistral, Google, and OpenAI have all published estimates for the resource consumption of a single interaction. The figures range from fractions of a watt to numbers comparable to the email example mentioned earlier, with water consumption varying from a few drops to an entire bottle. The challenge is that each company measures different parts of the process using different methodologies, making direct comparison difficult.
What is perhaps more telling than the numbers that have been published is what remains absent.
Google’s own reporting focuses on model usage, not training. Video generation—currently the most computationally intensive AI application—still lacks official figures. Independent researchers estimate that generating a single Sora 2 video may consume thousands of times more resources than a text prompt, but those estimates come from outside OpenAI. Mistral is the only company that has publicly shared figures covering the entire lifecycle, including training. According to its reporting, training consumed as much water as a Dutch family of four would use over more than 1,700 years. Whether similar proportions apply elsewhere remains unknown.
To me, that lack of a shared benchmark is the real story—not the exact numbers themselves.
Meanwhile, global demand continues to rise. The International Energy Agency (IEA) predicts that data center electricity consumption will roughly double by 2030, with AI acting as the primary driver. New regulations such as Annex XI of the European AI Act will require model developers to document this consumption, representing an important step toward standardized reporting. These requirements will be introduced gradually and do not currently apply to organizations that merely use the models.
For me, continuing to consume computing power without reflection is no longer a sustainable strategy, even if the impact of a single conversation appears small in isolation.
Curiosity, however, is not in conflict with efficiency or regulation. In fact, curiosity is what made these optimizations possible in the first place.
As a tester, I have learned never to assume that “quality will probably remain the same.” You investigate, measure, and only then draw conclusions. That same mindset revealed how much computing power was being wasted on polite filler and unnecessary explanations. Efficiency, too, is a claim that should be demonstrated rather than merely asserted.
That is precisely what Premium Software Quality means to me: not just delivering software that works, but taking responsibility for the path taken to get there—including the resources consumed along the way.
Continuing to develop with AI does not have to conflict with using resources responsibly. It is simply the same investigative mindset applied to a new tool.
Jorre van Munster
AI Lead and Test Specialist at Polteq
