Photo: 123RF
The AI industry needs to understand how it can be much more efficient with its energy use, the founder of an AI-powered carbon accounting tool says.
CarbonTrail founder and chief executive Tom Hallam told Afternoons training AI models like ChatGPT took a lot of energy and water.
So how does AI use energy and how much does it use?
Hallam said models like ChatGPT were trained by a set of data, like Wikipedia and large data sources, which was turned into probabilities - how likely it was that one word came after another.
If you were to write the word 'good' it would be more likely to be followed by 'morning' or 'day' than it would the word 'blue', he said.
"Now if you times that by every single sentence in Wikipedia and all of the training data that [Chat]GPT is using, it's vast, there's a huge amount to go through.
"A 2021 study suggested that the precursor to ChatGPT took about a month to train - a long time in computing terms - and in doing so used about 1300 megawatts of energy and about 3.5 million litres of water - which is insane, it's just huge numbers."
The models were trained in data centres, with racks upon racks of computers called servers which all have Graphics Processing Units (GPUs) inside them. These generated a large amount of heat, he said, and needed to be cooled using water that was pumped in.
He said the US Google data centre, in 2021, used an estimated 12 billion litres of water for cooling.
Once a model was trained and you asked it questions, there was an energy use as well, he said.
A lot of this came down to how clean the energy was, he said.
"If you're training the model in New South Wales, and I looked this morning, New South Wales it's 648 grams of carbon dioxide for every kilowatt hour that's used. In New Zealand this morning it was 120 grams so a massive, massive reduction because we've got such a strong mix of renewables here."
Awareness of this issue was growing, Hallam said.
Listen to the full interview
CarbonTrail was founded in mid 2022 and Hallam said it aimed to make it easy for businesses to understand, report and then reduce their carbon emissions.
"What we're saying is, we're not going to stop using AI, far from it, we really believe it's the way forward, but we're going to be moving our training and serving of those AI models into New Zealand data servers that are powered by clean energy and we're going to be doing our most 'polluting activities' when we know that the mix of energy is at it's greenest."
CarbonTrail used AI to analyse the spending side of an organisation and an algorithm was used to understand a transaction's emissions profile and carbon dioxide estimate, he said.
"Obviously AI's incredibly useful and we're on this amazing journey with AI across all different industries but because it was such an important part of our service, that AI categorisation... we had to, as part of our conscious go into our own supply chain... how do we understand what's going on into the goods and services that make up our business?"
They discovered all of the big AI providers and all the major cloud providers were not always as open as they could be about the actual carbon footprint or emissions from delivering their services, he said.
"So we set about starting to measure that."
Things were changing too, with large companies like banks now having to report their emissions to the government, he said.
"And we're starting to see the effects of that start to filter down the chain and little bit, and certainly closer to home Countdown have just announced that they're going to be encouraging their supply chain to be reporting their carbon emissions.
"So for the very first time, SMEs who maybe didn't have too much of an idea about this or there wasn't a particularly strong push are now being asked ok well if you're going to be sending products and services to us... you're now going to be encouraged to understand what goes into making those [products]."