For AI infrastructure, electricity is no longer an invisible utility behind an API. Grid capacity, rack density, cooling, carbon intensity, water, accelerator supply, and interconnection time now determine where software can run and how quickly it can scale.
The IEA estimates that data centers used about 485 TWh of electricity in 2025 and projects roughly 950 TWh by 2030, with AI-focused facilities growing substantially faster than the total. The U.S. Department of Energy reports that U.S. data center load could double or triple by 2028. These forecasts do not mean every application should optimize for carbon before reliability; they mean capacity planning can no longer assume that compute appears wherever software wants it.
A region can have available cloud APIs but constrained accelerator quotas, limited electrical interconnection, high local grid stress, cooling restrictions, or long lead times for new capacity. Architecture must model energy and infrastructure constraints before choosing active-active replication, retraining cadence, inference geography, or a promise of unlimited elastic scale.
Power Usage Effectiveness compares total facility energy with IT equipment energy. It is useful for cooling and facility overhead, but it does not reveal whether the IT load performs useful work, whether the local grid is carbon intensive, how much water is used, or how much embodied carbon sits in underutilized hardware. Uptime Institute's 2025 survey reports little movement in average PUE for a sixth consecutive year, while facility age, size, climate, and region hide substantial variation.
Software teams need a stack of metrics: facility PUE, Carbon Usage Effectiveness, Water Usage Effectiveness, hardware utilization, joules or watt-hours per request or job, carbon intensity by time and location, model quality, and an application functional unit. The Green Software Foundation's Software Carbon Intensity specification, standardized as ISO/IEC 21031:2024, frames emissions as operational plus embodied carbon per functional unit.
The cleanest computation is usually the computation not performed. Cache deterministic and reusable results, deduplicate jobs, batch compatible inference, right-size models, quantize where quality allows, use retrieval before a larger model, cap generated tokens, reduce retries, improve accelerator utilization, and retire idle replicas. These changes reduce energy, cost, and capacity pressure together.
Carbon-aware scheduling should come after efficiency and correctness. Moving a job to a cleaner region can backfire if it duplicates large datasets, keeps extra clusters warm, increases network transfer, misses accelerator utilization, or creates reliability overhead. Calculate the full system boundary rather than optimizing the region label.
| Workload | Hard constraints | Useful flexibility | Energy-aware architecture | Failure to avoid |
|---|---|---|---|---|
| Interactive inference | Latency, availability, privacy, model quality, regional capacity. | Model tier, batching window, cache, speculative execution, optional features. | Route by SLO and capability first; use semantic cache, dynamic batching, efficient model fallback, and token limits. | Sending users across continents for a cleaner grid while increasing latency, retries, and duplicate replicas. |
| Offline batch inference | Completion deadline, data residency, approved model, input availability. | Start time, region, parallelism, instance type, checkpoint cadence. | Shift to cleaner time windows or regions after accounting for data movement; pause at checkpoints when capacity tightens. | Missing the deadline because carbon became a hard objective instead of a bounded optimization input. |
| Model training | Accelerator topology, communication bandwidth, checkpoints, fault tolerance, experiment reproducibility. | Start window, cluster, precision, batch size, curriculum, early stopping. | Maximize accelerator utilization, schedule around capacity and carbon windows, checkpoint safely, and stop low-value experiments early. | Optimizing facility PUE while GPUs sit idle waiting for data, synchronization, or failed jobs. |
| Retraining and evaluation | Freshness target, release gate, representative data, compliance. | Frequency, sample size, model candidates, time and location. | Trigger on meaningful drift rather than fixed calendars; reuse features and embeddings; run staged evaluation before full retraining. | Retraining large models on schedule when data and product behavior have not materially changed. |
| Data pipelines | Data arrival, consistency, downstream SLA, residency. | Micro-batch interval, compaction time, region, storage tier. | Reduce scans, partition correctly, compact during favorable windows, and move compute toward data. | Moving terabytes to save a small amount of compute energy elsewhere. |
| Backups and maintenance | Recovery point, recovery time, durability, regulatory retention. | Execution window, compression, verification schedule, target tier. | Schedule flexible work during cleaner or less constrained periods while preserving recovery guarantees. | Deferring validation so aggressively that backups become unusable when needed. |
| CI and developer workloads | Feedback time for critical paths, reproducibility, security. | Optional matrix jobs, nightly tests, preview lifetime, runner location. | Cache dependencies, cancel superseded builds, hibernate previews, and shift exhaustive suites outside constrained peaks. | Keeping every branch environment and accelerator runner alive for convenience. |
The Green Software Foundation's Carbon Aware SDK supports temporal and geographical shifting based on carbon intensity. Google Cloud and Microsoft guidance similarly recommend shifting flexible workloads when requirements allow. The important phrase is “when requirements allow.” Carbon is one signal among SLO, price, queue time, capacity, data location, water stress, security, and business priority.
Use forecasts with confidence intervals and fallback behavior. A scheduler should know the latest acceptable start time, maximum relocation distance, data-transfer budget, approved regions, minimum model quality, checkpoint cost, and conditions under which it ignores the carbon signal. Forecast outages or stale intensity data must not strand production work.
Cloud customers rarely receive direct joule measurements per request. Use transparent proxies: accelerator-seconds by type, CPU time, memory allocation, storage and network bytes, utilization, provider emissions reports, facility metrics where available, and regional grid intensity. State uncertainty instead of manufacturing precision.
Attach energy and carbon budgets to products and workloads, not only infrastructure accounts. Report per functional unit such as successful inference, thousand processed documents, completed training run, or active customer. A falling total can hide declining demand; a falling per-unit score can hide rebound from rapidly growing volume. Track both.
When power or accelerator capacity becomes scarce, software needs graceful demand shaping: queue non-urgent jobs, lower model tiers, reduce optional context, pause batch work, reject low-priority experiments, extend deadlines with consent, or serve cached results. These are product and API contracts, not data center implementation details.
Expose execution classes such as interactive, deadline batch, interruptible, and best effort. Let callers declare quality floors, deadlines, residency, and whether results may be cached. The scheduler can then make legitimate tradeoffs rather than guessing which request is flexible.
I would build an energy-aware placement control plane joining workload contracts with accelerator inventory, queue predictions, cloud cost, grid intensity forecasts, PUE and water signals, data location, model benchmarks, and reliability policy. It would optimize within hard constraints and record why a placement decision was made.
The operations view would show useful work per accelerator-hour, energy and emissions estimates with uncertainty, cache avoidance, idle capacity, regional constraint, deadline risk, water exposure, and the marginal effect of model or scheduling changes.
Energy-aware architecture is not “run everything in the greenest region.” Reduce demand first, preserve hard requirements, expose workload flexibility, optimize across the full system boundary, and measure energy against useful outcomes. Electricity becomes an architecture constraint when software is capable of responding to it safely.
Article about data center energy as a software architecture constraint, including AI scheduling, caching, model routing, regional placement, carbon-aware workloads, water, and capacity.