Quoted from: Couillet, Romain, Denis Trystram, and Thierry Ménissier. “The submerged part of the AI-ceberg.” IEEE Signal Processing Magazine, September 2022.

The energy consumption of a single training run of the latest (by 2020) deep neural networks dedicated to natural language processing exceeds 1,000 megawatt-hours (more than a month of computation on today’s most powerful clusters). This corresponds to an electricity bill of more than 100,000 euros (figures in the millions of euros are sometimes found) and 500 tons of CO2 emissions – that is, the carbon footprint equivalent to 500 transatlantic round trips from Paris to New York. In comparison, the human brain consumes in a month about 12 kWh, i.e., a hundred thousand times less, for tasks much more complex than natural language translation. [Read more…]