The Environmental Impact of AI

Artificial Intelligence is pushing the limits of Moore’s Law. The number of transistors on processing chips increases roughly twice in magnitude every 18 months, resulting in more compute power being available in the market at a lower cost.

Demand for High Performance Compute (HPC) is increasing exponentially, especially with breakthroughs in generative AI technologies such as Large Language Models (LLM). These models rely on mathematical operations such as matrix multiplications performed in parallel on GPU technologies.

Training these models takes several days of HPC operations on thousands of GPUs — enough to burden finite resources such as fresh water and green energy supply.

So, what is the environmental impact and cost of AI?

How to measure emissions & environmental footprint

Let’s first understand how the environmental impact is calculated, and how a majority of emissions go unreported or unaccounted toward a company’s environmental footprint.

Emissions are categorized as scope 1, 2 and 3 according to the Green House Gas Protocol of 2001 and reflect a company’s responsibility toward their environmental footprint:

Scope 1

Scope 1 includes direct emissions owned or controlled by an organization as part of its operations.

For example, resources such as the energy and water consumed by GPUs and running data center operations for training the ChatGPT AI model directly are labeled as Scope 1.

Scope 2

Scope 2 includes indirect emissions as a consequence of generating energy that is used by a company. For example, the power company facility — such as a dam, solar power plant, or a wind farm — consumes energy to operate. The transmission and distribution company loses energy in the form of line losses.

Therefore, these energy consumption figures per unit of energy consumed by an end-user account for indirect or Scope 2 emissions.

Scope 3

Scope 3 includes everything else: indirect emissions that are not covered by Scope 1 and Scope 2, and resulting from the supply chain of a company are labeled as Scope 3. Examples include:

• The water and energy consumed to manufacture and supply GPUs that run AI models (upstream emissions).
• An end-user running queries on a ChatGPT instance on their personal computer.

A research survey of over 28,000 companies finds that 80% of carbon emissions are categorized as Scope 3 — and these are routinely unreported. That’s problematic, and it plays out in two ways:

• Around 66% of organizations planning to reduce their carbon footprints also leave out Scope 3 emissions from their future sustainability targets.
• The remaining third of organizations have no emissions target at all.

(Related reading: sustainable technology & using the Splunk Sustainability Toolkit to become more sustainable.)

Water utilization of AI models

Let’s start with water and the water utilization of large AI models. Water usage includes:

• Water withdrawal
• Water consumption

Water withdrawal is any fresh water that is extracted from a source (temporarily or permanently), used for a primary purpose (such as data center cooling) and then transferred to a secondary downstream usage system such as agriculture or municipal.

Water consumption refers to the difference between water extracted for withdrawal and water discharged to a secondary system after its primary use. Consumed water is discharged into the environment in a few ways:

• Evaporation
• Incorporation into land
• Otherwise made unavailable from the immediate water environment

Water withdrawal & consumption for ChatGPT

Now consider the 175 billion parameter GPT-3 AI model — the precursor to ChatGPT. This model consumes 5.4 million liters of water, of which 700,000 liters is attributed to direct Scope 1 water consumption. These include use cases such as cooling towers as a heat-rejection mechanism in the datacenter.

Off-site energy generation in thermoelectric power plants that produce 73% of utility scale energy in the U.S. attribute a Scope 2 water utilization. Water withdrawal is estimated at 43.8 liter/kWh and water consumption is estimated at 3.1 liter/kWh respectively.

(Here, Meta reports that its global datacenter systems consume 3.58 liter/kWh.)

Scope-3 water consumption is difficult to calculate accurately, especially since silicon chips are manufactured globally with a vast supply chain and limited visibility into the process. However, water recycling rates in the semiconductor manufacturing industry remain low for wafer plants (45 percent) and semiconductor plants (23 percent).

OpenAI uses Microsoft Azure datacenters for its ChatGPT products and the exact datacenter location for ChatGPT model training and inference is not reported. If we look at the combined average water consumption footprint for these technologies, the U.S. average stands at 3.142 liter per kWh.

• For model training, Scope-1 onsite water consumption is 0.708 million liter and off-site water consumption is 5.439 million liter.
• For inference, the model consumes around 500 ml of water for an average of 29.6 user queries.
• The total estimated energy consumption for GPT-3 was 1287 MWh. A detailed research report is available here.

For context, the combined water withdrawal of Google and Meta globally stands at 2.2 billion cubic meters (1.5 billion cubic meters in the U.S. or 0.33% of U.S. water withdrawal), which is roughly twice as much water usage as Denmark.

Electricity utilization of AI models

Additional research report also finds that the global AI market is expected to consume up to 134 TWh of electricity by the year 2027, with Scope-1 and Scope 2 water usage reaching 6.6 billion cubic meters.

At this rate, the report also projects the U.S. to host half of all AI workloads, which would make up to 0.7% of its total water withdrawal annually.