You open a chatbot on your phone and type something small, like a substitute for buttermilk. The reply starts appearing within a second or two, word by word, and nothing about it feels physical.
But your question has just left the phone, crossed a few networks and landed inside a building that could be hundreds of kilometres away.
In that building, racks of chips are doing arithmetic on your sentence. The building is wired to a substation, and the substation to power stations you’ll never see. What follows is that chain, one link at a time.
Where the prompt goes
Your text is tiny, a few hundred bytes. It travels over the internet to infrastructure run by the chatbot’s provider, which may be in a data centre far from you.
There, normal servers may handle routine work: check your account, apply safety filters, keep count of your usage. Then they pass the text to a different set of machines whose job is to run the model.
Those machines are the expensive part of the whole system, and everything else exists to keep them fed and running.
This is also why an AI service can feel instant while doing a surprising amount of work elsewhere. The same basic idea appears when you hand work over to AI instead of doing it yourself: the interface is simple, but the machinery underneath can be complicated.
Why ordinary servers aren’t enough
A language model is a huge collection of numbers. To produce each word of the reply, the machine has to push your text, plus everything it has already written, through those numbers, and do it again for the next word.
That is a very large amount of multiplication, but each multiplication is simple, and thousands of them can happen at the same time. Graphics chips (GPUs) were built for exactly that shape of work, which is why they ended up at the centre of all this. Google uses its own chips called TPUs for the same purpose. The general name for these is AI accelerators.
A web server loading a page is a different animal. It fetches a few things, assembles them and sends them off, and each request is small and independent. A chatbot request is heavier, and large models can be spread across many chips that have to keep talking to each other while they work.
Many calculations happening together
The useful trick is not that a GPU is magically “smarter.” It is that it can handle huge amounts of similar mathematical work in parallel.
You can see this in how the hardware is sold. NVIDIA’s GB200 NVL72 is a liquid-cooled rack holding 72 GPUs and 36 CPUs, wired together so they behave like one big system. Training a model takes it much further. Meta has described building two clusters of 24,576 GPUs each, with dedicated network fabrics between them, and used that design to train Llama 3. Training happens once, over weeks. Answering questions happens constantly, for as many people as show up.
Inside the building
A data centre is rows of racks, each about the size of a large fridge, and the racks are where the numbers get big. Most operators in Uptime Institute’s 2025 survey reported their highest-density racks below 30 kilowatts, while 10–30 kW racks are becoming more common.
An HPE listing for the GB200 NVL72 puts the system’s total power at 132 kilowatts and identifies 115 kilowatts for liquid cooling. The IEA, the international energy body, says that by 2027 a single advanced rack could have a peak power demand equal to 65 households, and that AI server power density rose 11 times between 2020 and 2025.
The computing is getting denser
The physical size looks familiar. The amount of computing and power packed inside it is what has changed.
Why it uses so much electricity
Part of the answer is that these chips run hard and nobody wants them idle, since they cost a fortune. Part of it is everything around the chips. Google published a breakdown for its own service: the AI chips account for 58% of a prompt’s energy, the host processors and memory another 25%, machines kept ready but idle 10%, and building overhead 8%. So a chatbot’s electricity isn’t only the chip doing the maths.
The chip is only part of the bill
Processors, memory, idle capacity and the building itself all add to the total.
People want a single number for what one question costs, and there isn’t one. Google’s estimate for its median Gemini text prompt, measured in May 2025, is 0.24 watt-hours, or 0.10 if you count only the active chips.
Sam Altman, OpenAI’s CEO, has said that an average ChatGPT query uses about 0.34 watt-hours. Those are company self-reports for particular models at particular times, and they measure different things. Longer answers cost more, and so do bigger models, longer documents and busier or quieter servers.
The IEA’s latest view is that a simple text query now typically uses less electricity than running a TV for the same time, but video, reasoning and agent-style tasks can use hundreds or thousands of times more per request. The energy per task keeps falling, while the tasks people ask for keep getting heavier.
This is Google’s measured median for Gemini Apps text prompts in May 2025, not a universal number for every AI request.
That distinction matters. A short text reply and a long reasoning task may both look like “one prompt” on screen, but they are not the same workload underneath.
Cooling
All that electricity ends up as heat. The IEA compares the heat from one advanced rack to about 30 gas boilers, and air blown across chips can’t carry that away. Newer racks run coolant through metal plates pressed against the chips, and the warm liquid is pumped out to equipment that dumps the heat outdoors, sometimes with electric chillers and sometimes by evaporating water.
The cost of all this shows up in a ratio called PUE, which is total site power divided by the power reaching the computers. Uptime Institute’s 2025 survey put the global average at 1.54, meaning roughly half a unit of extra power for every unit the computers use. Google reports 1.09 for its global fleet in 2025, so the spread between operators is wide.
Water use also depends heavily on the site, and Google’s figure of 0.26 millilitres per median prompt can’t be applied to every data centre.
Uptime Institute weighted average PUE
Google’s reported global fleet PUE
PUE is useful because it shows that the electricity bill is not only about computing. A data centre also has pumps, fans, cooling equipment, power conversion and other systems that keep the computing equipment operating.
Getting the power in
Electricity arrives over high-voltage lines to a substation, often built for the site. Transformers step it down, and batteries and diesel generators stand behind it in case the supply drops. Chatbot hardware is hard on this equipment, because the IEA notes that server load can swing by more than 50% of capacity within a second. Batteries are being installed partly to help manage those swings, and the IEA thinks 20–25 GW of battery storage could sit inside data centres by 2030. That is a projection, not a count.
Where it comes from
Beyond the substation it is just the grid, with whatever generates power in that region. The IEA’s outlook expects renewables to meet nearly half of the growth in data-centre demand to 2030, while natural gas and coal together meet over 40% of the additional demand, with small nuclear reactors becoming more important after 2030.
Some developers in the US, tired of waiting for grid connections, are building their own gas generation. The IEA says reliable onsite gas power means overbuilding the generators by 30% to 70% relative to demand, and estimates 15–27 GW of onsite natural gas may power data centres by 2030.
A data centre does not have a special kind of electricity. The physical electrons come through the local grid or an onsite source, while companies may separately contract for renewable electricity or other forms of supply.
Why this is now a grid problem
The confirmed part first. The IEA estimates that data-centre electricity use grew 17% in 2025 while total global demand grew 3%, and AI-focused data centres grew 50%. Its projection is that use roughly doubles from 485 TWh in 2025 to 950 TWh in 2030, accounting for around 3% of global electricity demand by that date.
That is a forecast, and it depends on chip supply, money and how people use these tools. The same report says the central projection could move as bottlenecks, investment and AI use change.
Huge numbers do not always mean huge everyday usage
A queue of planned power demand is not the same thing as power already being consumed. Some projects will be delayed, resized, cancelled or never connected. The number matters because it shows how much infrastructure developers are asking the grid to prepare for.
The global picture hides the local one. In the US, data centres account for nearly half of the demand growth to 2030. In Ireland, they took 23% of all metered electricity in 2025, up from 5% in 2015.
In Texas, ERCOT said in June 2026 that it was tracking more than 438 GW of large-load requests, with nearly 89% coming from data centres. Requests are not buildings, and plenty will never be built. Still, the queue shows how much load people expect to plug in.
Whether this pushes up bills is disputed, and the IEA’s answer is that it depends. In tight systems, new load can force expensive investment; in systems with spare supply, it can spread costs and even lower prices. Data centres also tend to oversize their grid connections at first, which makes planning harder. A shortage of high-bandwidth memory for chips is expected to last through at least the end of 2027, so growth is also being held back from the supply side. That wider chip-supply story is part of the same physical chain you can see in the RAM shortage affecting phones and laptops.
Software on a physical base
You type into a box on a phone, but the reply is made in a building full of chips, fed by cables from a substation, cooled by pipes, and paid for at the far end of a power line. The chat window is the only part of it that looks like software.
That is the part worth remembering. AI may arrive as a chat box, but behind that small box is a physical system getting larger, denser and more connected to the electricity network every year. The software is the visible part. The machines underneath it are the story you normally don’t see.
Sources & fact check
IEA, Key Questions on Energy and AI (2026)
Supports data-centre electricity demand, the 485 TWh to 950 TWh projection, AI growth, rack power density, battery storage, load swings and onsite gas.
IEA, Energy and AI (2025)
Supports the outlook for renewables, gas, coal and nuclear, and the US and global growth shares.
Google, Measuring the environmental impact of delivering AI at Google Scale
Supports the 0.24 Wh median prompt and its energy breakdown.
Google Cloud, Measuring the environmental impact of AI inference
Supports the 0.10 Wh active-chip figure and 0.26 mL water figure.
Hannah Ritchie, What’s the carbon footprint of using ChatGPT or Gemini?
Reports Sam Altman’s 0.34 Wh ChatGPT estimate.
NVIDIA, GB200 NVL72
Supports the 72-GPU liquid-cooled rack design.
HPE, GB200 NVL72
Supports 132 kW per rack and 115 kW for liquid cooling.
Meta Engineering, Building Meta’s GenAI Infrastructure
Supports the 24,576-GPU training clusters.
Uptime Institute, 2025 Global Data Center Survey
Supports the typical 10–30 kW rack range.
Google Data Centers, PUE
Supports Google’s reported PUE figure.
