In the past decade, the Machine Learning community has achieved breakthrough improvements on a variety of inference, prediction, and control tasks. Primarily, these improvements have been facilitated by an explosion in computational power and large clusters of machines working efficiently together. The cost of a 10% reduction in model error rate can often translate into a 1,000 fold increase in model size, and several orders of magnitude more energy being expended in training and running these eventual models. As data become ever more plentiful, and data scientists rely more and more on large state-of-the-art modelling algorithms, the question of the efficiency of learning per Watt of expended energy–and how we compare the ultimate utility of relative improvements in model accuracy–becomes ever more salient. Recent work in the Journal of Machine Learning Research by Henderson, Hu, Romoff, et. al. (JMLR, 2020, 20-312.pdf (jmlr.org)) provides among the first frameworks for measuring the carbon footprint of Machine Learning algorithms. The question of the sustainability of running large-scale computations and ML applications has also gained traction since conservative estimates of the AlphaGo winning model against Lee Sedol placed it at around 1MW of power consumption just during the match. Concerns have been raised during United Nations meetings, at major Machine Learning academic conferences, and among sustainability advocates. At a minimum, the community needs to be aware of energy consumption as an additional factor in the optimisation of large-scale models. Beyond that, the AI community should take an active leadership role in ensuring this science only has beneficial consequences for our planet and those who inhabit it.
At the AI for Good Foundation, all of our models and associated infrastructure are trained and run on machines powered by fully-renewable energy sources. Furthermore, we are seeking ways to engage the community in minimising energy consumption as an additional part of the puzzle at training and run-time. Engage with us, and help us to define sustainable AI for future generations of data scientists and researchers!
