Keynote: KubeCon + CloudNativeCon Europe 2024 in Paris - AI compute is growing exponentially, and somebody has to reconcile that growth with sustainability.
I joined three other engineers on the keynote panel to dig into exactly that problem - and where I think the responsibility actually sits.
I work on the platform engineering team at Intuit. This is a writeup of a panel I spoke on at KubeCon EU 2024 - my segment starts at 9:56 in the recording.
The panel#
“Keynote: Innovating Responsibly: How to Navigate Sustainability in the Era of Kubernetes” ran on Thursday, March 21, 2024 at KubeCon + CloudNativeCon Europe 2024 in Paris. Four of us shared the stage:
- Aparna Subramanian (VP of Engineering - Infrastructure, Shopify)
- Adrienne Jan (Chief Product and AI Officer, Scaleway)
- David Meder-Marouelli (Lead Architect, GMX, WEB.DE & mail.com)
- Todd Ekenstam (Principal Engineer, Intuit)
The through-line for the panel was “responsible innovation” - balancing the rapid growth of cloud-native and AI workloads against their environmental cost, from energy-efficient ARM hardware to what one of my co-panelists called “code sobriety.” My part focused on the piece I deal with every day: what it actually means to run a large Kubernetes platform efficiently.
The cloud consumer’s responsibility#
As a cloud consumer, I think the most important lever for sustainability is using your resources as efficiently as possible - and a big part of that is effective autoscaling.
At Intuit, that’s not an abstract concern. We run 100% of our services on modern SaaS infrastructure, processing 65 billion machine learning predictions a day for 24,000+ financial institutions, moving $560 billion, and handling 3.6 billion requests during peak season at 99.999% availability. We’re building an AI-native development platform on top of Kubernetes and cloud-native software, and using AI and data analytics inside the platform itself.
Load like that isn’t constant. We depend on autoscaling to handle the heaviest bursts, and just as much on scaling back down the moment that capacity isn’t needed anymore.
The over-provisioning trap#
Sizing and scaling workloads correctly is hard, and that difficulty breeds a “better safe than sorry” instinct. How do you really know you’ve sized a workload correctly for every condition it will ever see?
Faced with that doubt, teams often just throw resources at the problem - arbitrarily pre-scaling to a large number of pods, or a large number of nodes, “just in case.” It buys peace of mind, but it also means higher cost and higher resource consumption than the workload actually needs. That’s not sustainable, especially when better alternatives exist.
Autoscaling is hard to get right#
Kubernetes and the ecosystem around it already give you the capability to scale workloads automatically and dynamically. The gap isn’t capability - it’s that configuring those systems correctly is genuinely hard for application developers to get right on their own.
Capacity planning is fundamentally a data problem, and that’s exactly the kind of problem AI is good at. We believe AI can have a real impact here, helping teams be far more efficient with their computing resources without requiring every developer to become an autoscaling expert.
Building intelligent autoscaling at Intuit#
That’s what we’re building at Intuit: an intelligent autoscaling recommendation system that takes the guesswork out of sizing, reduces the burden on our developers, and helps ensure every workload has the resources it actually needs - no more, no less.
It’s a large engineering investment, and it isn’t easy. But I’d rather put that effort into innovation and optimization than simply buy more hydrocarbons.
Watch the video#
Watch the full keynote and the rest of the panel’s perspectives: KubeCon + CloudNativeCon Europe 2024 – Day 3: The Power of Sustainable Computing.
