As part of the recent Carbon Aware Internet project funded by RIPE NCC in 2021, the Green Web Foundation worked with the non-profit organisation Ember Climate to annotate every publicly available IP address on Earth with geolocated carbon intensity data. This work contributed to a collaborative response from the authors below, to a recent call for submissions by the IETF’s Internet Architecture board workshop on Environmental Impact of Internet Applications and Systems towards the end of 2022.
It was accepted – huzzah! You can read the submitted position paper below, and the full set of accepted papers at the above link.
Authors
Chris Adams(1), Stefano Salsano(2), Hesham ElBakoury(3)
(1) The Green Web Foundation
(2) Univ. of Rome Tor Vergata / CNIT
(3) Consultant
Abstract
In this position paper we first review the concepts of Carbon awareness and of its application to the ICT industry with Carbon aware computing. We focus on the need of providing accurate and granular attribution of carbon footprint and energy consumption to ICT users and applications. By granular we mean an attribution over relatively short time scales (e.g. hours and minutes…). Starting from this requirement we focus on the “networking” component of the ICT applications. The challenges of introducing carbon awareness in networking are discussed and a conceptual model for Carbon Aware Networking is depicted. To address the requirements of accurate and granular attribution, our position is that it should be possible to dynamically collect carbon related information using packets in transit. We focus on IPv6 networks and propose protocol extensions that can support Carbon Aware Networking within a framework called EIP (Extensible In-band Processing). This proposal to support Carbon Aware Networking with EIP is still in the early maturity phase, and we identify the needs for further work at a multi-disciplinary level.
“Carbon awareness” – an introduction
In 2020, the ITU issued a press release stating that to comply with the Paris agreement, the ICT industry will need to reduce IT carbon emissions by 45% by 2030 [1], which amounts to an average reduction of around 7.5 % annually until 2030. As we draw to the end of 2022, it looks like the annual reductions will now have to be greater than this to meet the 2030 goal, and improvements in efficiency alone will not be enough to achieve these reductions, as the effect of Moore’s law wanes. With this in mind, we will need to decarbonise the energy computing relies on too, and adjust our systems to match a grid that has predictable, but varying amounts of clean energy available.
What is carbon awareness?
Carbon awareness is a property of a system that describes its ability to adjust its own behavior in response to changes to the carbon intensity of the electricity it consumes. This can reduce the environmental impact of the system by making it better at running on cleaner energy. It can increase the power drawn from lower carbon power sources when there is an abundance of clean energy available, and it can reduce or eliminate power drawn from the grid when fossil fuels make up the majority of generation powering the electricity grid.
Carbon awareness has economic implications too. The cost of renewable energy and battery storage has fallen nearly 10-fold in the last ten years [2], and when there is an oversupply of renewable energy on the grid the wholesale power cost of electricity often falls dramatically, or sometimes even goes negative [3] [4]. Because of this, if a system can scale its power demand up and down to match this supply of greener, cheaper energy, it can significantly improve its running costs too.
Carbon aware computing
In multiple sectors it is commonplace to take advantage of the changing prices of electricity to shift work through time, to save money and carbon emissions. HVAC (Heating, ventilation, and air conditioning) systems in large buildings often take advantage of cheap overnight power to generate ice for cooling during the day [5]. Elsewhere, companies like Octopus Energy have pioneered “agile tariffs” for EV owners, to time-shift when they charge their vehicles in return for favorable, or even negatively priced tariffs [6].
Large technology firms now also do the same with computing jobs that are important, but not time sensitive to save money and carbon in the same way. In early 2020, Google began publicly talking about how they move computing jobs temporally, through time, scheduling them to run when supplies of cheap, green energy are abundant [7]. A year later, they announced they had begun moving computing jobs geographically too, through space, scheduling the work to run in distant different datacentres that were enjoying an oversupply of low carbon energy [8]. Other firms like Lancium have gone further – they deploy small datacentres on-site to wind farms and solar facilities, purchasing excess electricity that can not be economically fed into the grid, at extremely low prices [9]. This allows them to pass savings on, offering computing at costs up to 90% lower than larger cloud providers to customers who have workloads that can be paused, or delayed. More recently, in simulations based on existing workload patterns shared by Google, and using their same infrastructure, Weisner et al have been able to demonstrate carbon emission reductions of over 20% through thoughtful scheduling of work [10].
Carbon aware networking, where similar ideas are applied to reduce the carbon footprint of network transfer are less developed. Globally, while the energy use is more diffuse, the sum of power drawn from networks worldwide is estimated to be around one and a half times that of datacentres [11], meaning that innovations at this layer may have substantial impact in terms of decarbonisation.
Earlier in 2022, the Green Web Foundation published open source grid intensity software libraries [12], and used open carbon intensity data published by non-profit Ember Climate [13] to annotate every public IP address globally with carbon intensity data [14, p. 2], making it trivial to query the carbon intensity of any public node on the internet. Why do this? The intention was to lower the barriers of entry to adopting these techniques – and the thesis was similar to this paper – that when accurate carbon intensity information is freely available, and combined with existing geolocation information for the internet, many ideas successfully applied for carbon aware computing can work to optimize the carbon intensity of the networking layer too.
For any system to be carbon aware however, you need to be able to evaluate the carbon intensity of electricity.
Evaluating and certifying the carbon intensity of electricity
In most parts of the world, the amount of electricity produced by a given state or country is tracked by various regulatory agencies or similar entities, along with the kind of generation used for generating the electricity – be it from coal, gas, or renewable sources.
Where this renewable energy generation is tracked, and reported, a regulator or other entity, often referred to as an issuing body, recognises this generation, by issuing a quantity of certificates, known as environmental attribute certificates, linked to the quantity of generation from these renewable sources. These certificates are used to independently confirm that energy generated and sold came from renewable sources that same year.
These certificates are either ‘bundled’ with the power itself, to be sold as green power, or in some regimes, sold separately onto a market where other energy generators can purchase the right to claim that a corresponding quantity of energy generated is green. In this scenario, the ‘greenness’ is treated like a fungible, transferable property.
In the energy sector this certificate-based approach is used to support policy interventions to green the grid. One way is mandating through policy that a given percentage of electricity in a state or region must come from renewable sources, creating a market for greener electricity that is confirmed by these certificates. A good example is in California’s Renewable Portfolio Standard [15], where 50% of the energy is now mandated to come from renewable sources by 2030.
Historically, these systems primarily work on an annual reporting basis, where you might try to match the quantity of energy used over a given year with a given amount of renewable energy certificates (be they from ‘bundled’ sources, ‘unbundled’ sources, or both). Unfortunately, this can lead to cases where the ‘green’ attributes that have been unbundled from clean solar power can end up being matched to energy usage that took place at night, when the sun is not even in the sky, undermining any claims of energy being green.
From annual certificates of green energy to publicly addressable, hourly certificates with URIs
As a response, a newer standard, named Granular Certificates, has been developed by the standards body EnergyTag [16]. Under this scheme, certificates are bound not just to a given year, but to a specific time of day, and can only be matched to energy used at the same time of day.
Certain times of day are more difficult than others to decarbonise because they often rely on different technology, like batteries, geothermal power, and so on. At the same time, a growing number of deep-pocketed buyers of energy are looking for ways to demonstrate their climate commitments, by visibly supporting these emerging technologies. Both Microsoft [17] and Google [18] have now made public commitments to run all their digital infrastructure on power where they can demonstrate such hourly matching by 2030, and both are actively involved in the development of the Energy Tag standard.
Finally, another interesting feature of Granular Certificates is that they have been developed alongside an open API spec, for any provider to make certificates information about who owns them publicly addressable at known URIs. If we already use TLS certificates and Certificate Transparency [19] to help us trust that a domain points to the infrastructure we think it does, it is not a huge leap to imagine using similar ideas to be able to trust that various nodes of the Internet really are running on renewable energy, by linking these publicly addressable certificates to these same specific nodes in a network.
The need to collect carbon footprint of networks with accurate attribution
Most of the time, the majority of an organization’s environmental impact comes from its supply chain. While there are increasingly well-established ways to attribute the impact associated with computing along a supply chain, quantifying and managing the emissions related to network transfer has historically been much harder, as so many intermediate actors are involved in supporting connections between two or more parties.
At the same time, increasingly stringent reporting requirements are driving organizations to accept more responsibility throughout their supply chain for carbon emissions. For this to be possible, they need to know who to attribute emissions to, and how to do so.
At present, the most popular methodologies use a top-down model, taking the global figures for sector energy use, and figures for the total amount of transferred bytes, then scaling them back down based on measured usage to arrive at a figure, in terms of energy per gigabyte of transfer, or energy per hour of a link in use [17]. This can provide us with a single number, but it does not tell us much about how to attribute responsibility along the supply chain. If one part of the network is powered by significantly ‘dirtier’ energy than others, and others are making efforts to decarbonize in line with the science, then these differences are not reflected in the model. Faced with this, organizations have opted to simply leave out the carbon footprint of network transfer entirely in their own emissions reporting, like in Netflix [20] in their report for 2021, amongst others.
Instead of relying on top-down models, and dubious attribution, we propose an alternative approach, that relies on direct, bottom up approaches where the resource footprint is calculated on a packet by packet basis. As packets traverse the networks, they would accumulate information about their resource footprint in packet headers, somewhat like stamps in a passport as a traveler traverses the globe.
While we outline the enabling network technologies below in more detail, it’s worth noting that many of the technologies already exist and are in production. IPv6’s extensible headers support this packet by packet reporting idea, and technologies like P4 [21] and eBPF [22] support the introduction of new processing logic into the routers without needing to replace them. Open data exists for both geolocation of nodes, and attribution of carbon intensity to a location at coarse grained level, as a way to provide a minimum level of information globally. Tools like Granular Certificates provide paths for more fine grained annotation of nodes with information of known providence.
Carbon Aware Networking
The need for introducing carbon awareness in networking is discussed in [11], which provides an excellent introduction to the main challenges to be solved. The authors of [11] identify some directions of work to address the challenges and explicitly set out a “Call to Action” for the networking community to contribute to the goal of reducing the carbon footprint of networks.
We are aligned to this vision and Call to Action. We provide our views of goals, challenges, and gaps, largely overlapped with [11].
In Figure 1 we represent a high-level conceptual model of the processes involved with Carbon Aware Networking. At the top of the model, there is the evaluation of the energy consumption and carbon intensity of the network devices. The next step is the collection of this information. Then comes the attribution to users, services and applications. The attribution process may lead to accounting, charging or to the creation of certificates. At the bottom of the model, the process of network optimization and carbon footprint reduction is depicted. This process receives inputs from the other processes as shown in the figure. The model is very general, and it is only meant to introduce and discuss the issues and challenges that still need to be addressed for Carbon Aware Networking.
Figure 1: An high level logical model for Carbon Aware Networking
A set of issues is related to the evaluation of energy consumption and carbon-intensity of network devices. This includes the measurement of the energy which is directly consumed by the device, but should also take into account the “external” energy consumption which is associated with a device (e.g. for cooling of the building). Evaluating the “external” energy consumption and associating it to a number of devices is a challenge. Another challenge is how to evaluate the carbon-intensity of the consumed energy. Assuming that these challenges are solved, the network device should support the collection of this information, which is needed by the other logical processes. Hence, we identify these high-level requirements:
- Network devices should be able to report their real-time or near real-time energy consumption.
- Network devices should be able to report the carbon-intensity of consumed energy.
Having a proper attribution of the carbon footprint is a goal in itself and it is a fundamental input for the optimization and global reduction of carbon emission. Hence we state that:
- It should be possible to evaluate the energy consumption and carbon-intensity at granular levels, e.g. per user, per application or service and with adequate time granularity (up to real time values).
With this granular level evaluation, it is possible to properly attribute responsibility for energy consumption and carbon footprint to users, or services/applications. The result of the attribution process can be used for accounting or charging purposes towards users, or to generate certificates. Unfortunately, achieving a granular level evaluation is very difficult in networks for a number of reasons: in general, a service/application spans multiple network domains, the networking resources are shared among a huge number of users/services/applications, the energy consumption of networks is composed of a fixed part and of a part that depends on the utilization, making very difficult to properly associate it to users/services/applications.
As shown in Figure 1, the information that is collected related to Energy consumption and Carbon Footprint of devices and that is further processed for proper attribution can be used as input to the optimization process that aims at reducing the carbon footprint of the network. This conceptual process can be realized in many ways operating at very different time scales and considering different granularity in the aggregation of the information. In [11], the concept of Carbon Aware routing is discussed, which consists in using the carbon footprint as the cost metric in routing protocols. This can be applied in distributed routing protocols or in a centralized routing solution using a Path Computation Element (PCE).
In the next section, we focus on the aspect of information collection. In order to support the requirement of granular level evaluation, we propose a solution based on dynamically collecting information using packets in transit.
Using EIP to collect information from network devices
EIP – Extensible In-Band Processing
EIP (Extensible In-Band Processing) [23], [24] has been proposed to extend the functionality of the IPv6 layer, supporting the requirements of future Internet services. With EIP, IPv6 nodes can read, write and process EIP information in packet headers to support different use cases (e.g. advanced monitoring, semantic routing, deterministic networking, slicing…). Each use case has its own specific architectural aspects and protocol details, while EIP provides a generic framework. In particular, EIP defines a single IPv6 hop-by-hop option that can be further specialized to support the specific use cases.
EIP is proposed for “limited domains” [25]. In a limited domain, we can assume that all IPv6 nodes (hosts, routers, middleboxes) are able to process the EIP header, or at least they can ignore it and forward the packet towards the destination. With EIP it is possible to collect information from intermediate nodes. Moreover, the information contained in the EIP header can influence the processing and forwarding of the packet, so the use of EIP is not limited to collecting information.
Using EIP for Carbon-Aware networking
We start from the assumption that network devices are able to evaluate and report their energy consumption and carbon footprint. We propose to collect this information by packets in transit using the EIP header. Once the information is in the packet, it can be processed so that it can be associated with a flow and in turn with the users and/or services that are sending/receiving the flow packets.
The detailed scenarios and procedures for carbon aware networking have not been defined yet. At the current state of the research, we can envisage different scenarios and for these scenarios we are designing a corresponding EIP header format. A possible scenario is to accumulate the carbon footprint associated with a packet, by summing up the contributions of all crossed nodes. In this scenario, all crossed nodes read the carbon footprint associated with the travel of the packet up to the current node in the EIP header, then they sum it with the carbon footprint of the node (which should be evaluated in real time) and write back the result in the EIP packet header. Another usage scenario is to explicitly analyze the contribution of each crossed node. This requires that the crossed nodes add their current carbon footprint in the EIP header. At the exit of the domain, the EIP packet header will contain the array with the carbon footprint of all the crossed nodes.
In both usage scenarios (accumulated carbon footprint or array of measures), at each hop the carbon footprint can represent the total instantaneous carbon footprint of the crossed device (per time unit), or the device can estimate a “per-packet” footprint, dividing the total instantaneous footprint by the current aggregate incoming packet rate of the device. Moreover, in addition to carbon footprint, it could be useful to evaluate the consumed energy (or equivalently the carbon intensity), the EIP header definition should also support this collection.
The EIP framework is based on the definition of specific Information Elements for the different use cases. In particular, for Carbon Aware Networking we defined a Carbon Aware Information Element that in turn can be used to carry the relevant information according to the scenario to be supported. In particular, a field called “Content type” can specialize the Carbon Aware Information Element to the different scenarios. Figure 2 and Figure 3 provide a snapshot of the current definition for carrying respectively the accumulated carbon footprint and the array of carbon footprint of all current nodes. Different values of the “Content type” field can also be used to differentiate between the different interpretations of the carbon footprint (total instantaneous footprint or “per-packet” footprint). When the array of Carbon Footprints is used, another EIP Information Element called “Node selection and identification” [26] can be used to record the identifiers of the crossed nodes that have reported their footprint.
Figure 2: Carbon Aware EIP Information Element (Accumulated Carbon Footprint)
Figure 3: Carbon Aware EIP Information Element (Array of Carbon Footprints)
Note that the collection of carbon footprint information does not need to be performed on every packet, as doing so would generate an intractable amount of information. A proper sampling approach needs to be devised, and this is an important aspect for further research.
Processing the collected information
Collecting the information in the packets in transit is only a part of the solution. We need to design the mechanisms and the infrastructure to process the collected information. Considering the EIP architecture [23], the path through an EIP domain ends with an Edge node that will extract the EIP header including the Carbon Aware Information Element. This information needs to be forwarded to a monitoring server or more in general to an analytics system. A possible solution for the monitoring/analytic infrastructure is the so-called TIG stack [27], where TIG stands for Telegraf, InfluxDB, and Grafana. As shown in Figure 1, the information collected from the EIP headers is the input to the attribution process and it can be used for the network optimization process. Therefore, these processes will interact with the monitoring/analytic infrastructure to retrieve the information they need. The design of the attribution process, i.e., how to associate the packet level information with the different users and/or services using the network is an open challenge. Another important research challenge is how to use the collected information for the optimization of the carbon footprint. When a centralized routing solution is used, the challenge is to design a PCE (Path Computation Element) capable of using the information retrieved from the monitoring/analytic infrastructure to optimize the network carbon footprint.
Conclusions
We started analyzing the current status of carbon aware computing. We observed that it is very important to focus on reducing the carbon footprint of networks, moving to carbon aware networking. The current approaches for evaluating the carbon footprint of the networking component of ICT services are far from satisfactory as they are very coarse in the attribution of carbon footprint. We advocated the need for an accurate and granular attribution of the carbon footprint, i.e. an attribution over relatively short time scales (hours or minutes).
We introduced a reference conceptual model to identify the different processes that need to be combined to fully support carbon aware networking. We analyzed the model, highlighting several challenges to be solved.
In order to address the requirement of accurate and granular attribution of carbon footprint, we proposed a solution based on the collection of information from network devices using packets in transit. In particular, we defined protocol extensions to IPv6 for Carbon Aware Networking within a framework called EIP (Extensible In-band Processing). Defining the protocol extensions to collect the carbon footprint information from the devices is only a part of the solution. The main goal of this work has been to identify the several open research challenges associated with the dynamic collection of carbon footprint at packet level. We hope to stimulate discussion and contributions from the community, and we observe that help is needed not only in networking aspects but also in energy engineering, energy management, economics and regulatory aspects.
References
[9] ‘Lancium’. https://portal.lancium.com/about/pricing
[22] ‘eBPF – Linux Foundation Project’. https://ebpf.foundation/ (accessed Oct. 30, 2022).
[25] B. Carpenter, ‘Limited Domains and Internet Protocols’. RFC 8799, 2020.