Measuring and Mitigating AI Carbon Intensity
The science is clear: the World Meteorological Organization reported that global temperature has a 50% probability of exceeding 1.5°C of warming within the next five years. Scientists have set this as the ceiling for avoiding catastrophic climate change; when we reach this as a long-term threshold, we can expect unprecedented disruptions to human quality of life and our supporting ecosystems. We must do everything possible to reduce our carbon footprint.
AI has the potential to accelerate progress; for example, by integrating renewable energy into an electricity grid or reducing the cost of carbon capture. At the same time, the technology itself needs to be sustainable. We don’t have an accurate measure of AI’s overall carbon footprint because the infrastructure to track and report this is still in its infancy. While we build this, we are committed to heeding the warnings set forth by the IPCC report—the time to act is now.
Rapid growth in technologies such as Machine Learning has provided people around the world with unprecedented access to computational power: the computational demands of these workloads can incur a high energy cost. As a result, research has increasingly focused on how to ensure AI models use computing and energetic resources more efficiently. Recently, Microsoft has made energy consumption metrics available within their machine learning platform to help users to understand the hidden costs of their workload. Energy translates to a real-world carbon footprint: so-called operational carbon emissions arise when the source of electricity is not carbon-free, meaning it factors in the carbon intensity of the grid that powers the data center.
The carbon intensity of a grid is sensitive to small changes in carbon-intensive generation, and can vary by both location and time. Each region incorporates a different mix of energy sources, so can vary widely. Carbon intensity varies by hour, day, and season due to changes in electricity demand, low carbon generation (wind, solar, hydro, nuclear, biomass), and conventional hydrocarbon generation. As a result, there are many opportunities to shift computing resources to capitalize on these variations: this is known as carbon-aware computing.
There are certain decisions that users can make to make their workloads more carbon-aware, such as choosing a geographic region or deciding when to run a training job. Knowing what actions are possible, and what impact they have, will help users make informed decisions on how to reduce the carbon footprint of their workloads. Organizations are mobilizing: the Green Software Foundation is a cross-industry consortium that is defining a set of people, standards, and tools to enable this.
“You can’t manage what you can’t measure” — Peter Drucker
Without a consistent framework to measure operational carbon emissions on a granular basis, users and cloud providers cannot take effective action. To address this, we at Microsoft & AI2, with co-authors from Hebrew University, Carnegie Mellon University, and Hugging Face, have applied the Green Software Foundation’s specification for measuring Software Carbon Intensity (SCI) to measure the operational carbon emissions of Azure AI workloads by multiplying the energy consumption of a cloud workload by the carbon intensity of the grid that powers the data center, using data provided by WattTime. The SCI uses what is known as a “consequential” carbon accounting approach, which aims to quantify the marginal change in emissions caused by decisions or interventions, or actions.
To understand the relative SCI of a wide range of ML models, we compared 11 different experiments against estimates of equivalent sources of emissions per the United States Environmental Protection Agency. In this chart, blue bars represent variation across regions.
Note: the 6 billion parameter LM is only trained for 13% of a full run; a full run would emit about an order of magnitude more emissions than shown. Therefore, larger models could have the approximate carbon footprint of a rail car of coal.
We then assessed several different actions that a user could take to minimize their SCI using carbon-aware strategies. Among these, we found that choosing the appropriate geographic region plays the largest role; selecting it can reduce the SCI by almost 75%. If other factors such as latency and cost are equal, we encourage users to reference this chart while choosing a location for their computing resources.
Additionally, the time of day plays an important role: depending on the duration of a workload, there is a significant reduction potential to capitalize on diurnal variations in carbon intensity. For shorter training runs, we find reductions greater than 30% in multiple regions, and up to 80% in regions that have high renewable energy intermittency. For significantly longer workloads, the reduction is less than 1.5%. This carbon-aware scheduling has the potential to extend beyond training into other deferrable ML workloads such as batch inferencing or data processing.
To further reduce the operational footprint, workloads can be dynamically paused when carbon intensity is high, and resumed when emissions are low. This capitalizes on the daily variations in carbon intensity and optimizes for carbon emissions within a predefined window. Within the boundary of doubling the original duration, we found that the shorter workloads had the least reduction potential (less than 10%), while the larger workloads actually exhibit the largest decrease (about 25%).
It is important to note that these reductions and operational carbon calculations are for a single training run: any measure of AI’s overall carbon footprint would need to account for the full lifecycle of an ML model, starting with the initial exploratory training phases all the way through hyperparameter tuning and deployment, and monitoring of the final model. Future research is needed to inform an Operational Lifecycle Analysis (OLCA) for an ML model.
Fortunately, major cloud providers (including Microsoft) are already powering their cloud computing data centers with 100% carbon-neutral energy through market-based measures such as Renewable Energy Credits (RECs) and Power Purchase Agreements (PPAs). However, purchasing clean energy is not the same as running on clean energy; as developers and users, we must be conscious of when and where we run our cloud workloads to minimize our environmental impact. Many groups across Microsoft are taking action; for example, Windows updates are now scheduled during times of lowest marginal carbon intensity.
As organizations and developers mobilize, there is a need for centralized and interoperable tooling to enable this at scale. The Green Software Foundation’s Carbon-Aware Core SDK is a nascent open-source project to build a common core that is flexible, agnostic, and open: this allows software and systems to build in native carbon-aware capabilities.
As we outlined in our paper ‘Measuring the Carbon Intensity of AI in Cloud Instances’, cloud providers providing software carbon intensity information in an actionable way will empower developers & users to reduce the carbon footprint of their AI workloads. This requires interoperable measurement tooling; only once this is built can effective carbon management strategies be built. To get started, learn more about the Green Software Foundation, and start contributing to open-source projects through the Green Software Foundation’s GitHub to build carbon-aware capabilities into your applications. This potential extends beyond machine learning workloads, and we look forward to sharing additional developments in the coming months.
Follow @allen_ai on Twitter and subscribe to the AI2 Newsletter to stay current on news and research coming out of AI2.
