My AI runs on sunlight, mostly

Notes from a spare room, a second-hand graphics card, and a sustainability practice.

Sunday afternoon. The sun is on the panels and there is a graphics card in the next room drawing power off them to think.

That is the honest picture of it. A second-hand RTX 3090, 24 gigabytes of memory, running a language model small enough to live on my own hardware. I ask it something, it answers, and on a clear day the electricity doing the answering came off my roof rather than out of the wall. No data centre. No request flung across the country to a warehouse full of servers and cooling towers. Just a box in the corner and the sun.

I did not set out to make a point. I set out to stop paying rent to someone else’s cloud. But once the thing was running I could not shake the shape of it, because I have spent the last few years on the other side of my desk doing energy and sustainability work, and the shape was familiar.

Here is the shape. Fossil fuels are centralised: a handful of enormous plants, vast capital, output pushed out along wires to the rest of us. Renewables, at their most interesting, invert that. A panel on your roof, a battery in the garage, generation sitting where the demand is. Power moves from the few to the many, from heavy to light, from somewhere else to right here.

Cloud AI looks a lot like the power station. A few companies, enormous capital, warehouses of chips drinking electricity and water, and the rest of us sending our questions off to be answered somewhere we will never see. My little rig looks like the rooftop panel. Same move. Centralised to distributed. Heavy to light.

It is a satisfying parallel. It is also, if I am careful, only half true, and the half that is false is the interesting half.

Decentralisation is not the win. I want to say that plainly, because every recycled tech-utopian pitch of the last decade said the opposite and most of them were selling something. Not everything wants to be distributed. Training a frontier model is centralised to the bone: it needs capital, chips and power at a scale no spare room will ever hold, and pretending otherwise is a fantasy. And renewables themselves are not the distributed story we tell about them. I live in South Australia, where the grid runs past three-quarters renewable and is closing on net 100 per cent by 2027. It got there on wind farms, grid-scale solar and some very large batteries, alongside the highest rooftop uptake in the world. The socket in my wall is nearly as clean as the panel on my roof. Decentralisation did not clean this grid on its own; scale did most of the lifting.

And the counter-intuitive bit: a hyperscale data centre, run well, is often more efficient per unit of work than my box in the corner. Purpose-built cooling, batched requests, hardware kept busy. My 3090 idles between questions and wastes what it idles. On raw efficiency, the power station wins. It usually does. That is why we build them.

So if the win is not decentralisation, what is it?

Two things, and neither is romantic.

The first is that the tools finally exist. Open-weight models you can download and run yourself, good enough for real work, small enough for consumer hardware. That is new. Two years ago the local option was a toy. Now an eight-billion-parameter model on a second-hand card handles most of what I throw at it. The distribution became possible not because someone believed in it, but because the engineering caught up.

The second is the one I keep coming back to. The footprint problem is less about where the model runs than how big a model you reach for. Most of us, most of the time, are using a cathedral to answer a question a shed could handle. We send trivial requests to the largest model available because it is there and it is easy, the way you might leave every light in the house on because the switch is a reach.

The move is not lightweight. It is right-weight.

Match the model to the task. A small model on your own hardware for the ninety per cent that is ordinary; the big centralised machine kept for the ten per cent that earns it. Do that and the footprint takes care of itself, whether the electrons came off your roof or not.

The panel on my roof is a nice touch. It makes a good photo, and it is real: on a Sunday like this one my questions do run on sunlight. But it is not the argument. The argument is quieter. Stop reaching for the cathedral.

Most of the time, the shed will do.