When Not to Use Cloud Computing: Latency, Data Sovereignty, and Total Cost Reality
The technology industry has a tendency to treat cloud computing as the default answer to every infrastructure question. Consultants recommend it. Job postings require it. Conferences celebrate it. That enthusiasm, while mostly earned, has created a pattern where cloud is the answer before the question is fully formed.
The honest reality is that cloud computing is not universally superior to on-premise infrastructure. There are specific, well-defined scenarios where on-premise or hybrid approaches outperform cloud on cost, latency, regulatory compliance, or operational control. Understanding those scenarios makes you a better architect than assuming cloud always wins.
When Latency Requirements Are Absolute
Cloud computing adds network hops. When your application calls an AWS RDS database, the request travels from your application server (running in the same region, often in the same availability zone) over AWS’s internal network to the database. That round trip is typically 1–5 milliseconds.
For most applications, that is invisible. For some, it is unacceptable.
High-frequency trading: Electronic trading systems execute thousands of orders per second. A difference of 100 microseconds — 0.1 milliseconds — determines whether a trade executes before or after a competitor’s. HFT firms co-locate physical servers in the same building as exchange matching engines, connected by fiber patches measured in meters. Cloud networking cannot approach this; even the “ultra-low latency” marketing from cloud providers refers to milliseconds, not microseconds.
Industrial control systems: Manufacturing equipment, power grid controllers, and process automation systems often have hard real-time requirements. A valve controller that needs to respond within 10 milliseconds to a sensor reading cannot accept variable network latency from a cloud endpoint. These systems run locally, close to the hardware they control.
High-performance computing with tight coupling: Some scientific computing workloads — molecular dynamics simulations, computational fluid dynamics, weather modelling — require nodes to communicate with each other thousands of times per second during a single computation. The inter-node latency in cloud environments (even with cluster placement groups) exceeds what dedicated InfiniBand networks in physical HPC clusters achieve. If the MPI communication overhead exceeds 5–10% of compute time, local infrastructure often wins.
Latency by Infrastructure Type---------------------------------Local memory access: ~0.0001 msLocal SSD: ~0.1 msSame-DC network (physical): ~0.5 msCloud intra-region: ~1–5 msCloud inter-region: ~30–100 msCross-continent (internet): ~100–300 ms
HFT requirement: < 0.01 ms --> cloud cannot competeIndustrial control: < 10 ms --> cloud is marginal, riskyWeb application: < 200 ms --> cloud is excellentData Sovereignty and Regulatory Constraints
Data sovereignty is the principle that data is subject to the laws of the country or jurisdiction in which it is collected, processed, or stored. As privacy regulations have proliferated globally, data sovereignty has become a significant cloud obstacle.
German financial institutions under BaFin guidance: German banking regulators require that certain categories of customer data remain within Germany, with documented access controls, and with the ability to audit every access. While AWS, Azure, and GCP all operate German regions, the regulatory question is not just geography — it is about the provider’s access to the data, the legal jurisdiction of the provider’s parent company (US-headquartered for all three major providers), and the potential applicability of US legal processes (CLOUD Act, FISA orders) to data stored on those systems. Some German banks have concluded that the regulatory risk is not worth taking for certain data categories.
Chinese data laws: China’s Data Security Law and Personal Information Protection Law impose strict requirements on how data about Chinese citizens is collected, processed, and transferred. For multinational companies with Chinese operations, separating Chinese-user data from global infrastructure is often legally necessary, not optional. Building on a Chinese hyperscaler (Alibaba Cloud, Tencent Cloud, Huawei Cloud) may be required.
Healthcare with strict HIPAA interpretations: While cloud providers offer Business Associate Agreements (BAAs) that cover HIPAA compliance, some healthcare providers — particularly those dealing with mental health records, HIV status, or substance abuse treatment — interpret their obligations more conservatively and prefer on-premise storage for the most sensitive record categories.
The Total Cost Reality for Stable, Large Workloads
Cloud pricing is designed around elasticity: you pay more per unit because you are paying for the option to scale and the operational convenience. For workloads that are large, stable, and continuous, that flexibility premium may not be worth paying.
The break-even calculation depends on several variables:
-
Compute utilisation: Cloud VMs are priced assuming variable utilisation. If you run a fleet at 80% CPU utilisation continuously for three years, reserved instance pricing is competitive. But if you run at 80% continuously and know you always will, dedicated hardware may cost 50–60% less.
-
Storage at scale: Cloud object storage at multi-petabyte scale (common in media, genomics, satellite imagery) costs 25 per terabyte per month on S3 standard. Enterprise-grade on-premise object storage hardware amortised over five years costs roughly 4 per terabyte per month. For 5 petabytes of data that is not being accessed frequently, the annual difference is 1.4 million.
-
Egress costs: Cloud providers charge for data leaving the cloud. AWS charges 9,000 per month in egress alone, before compute or storage costs. A CDN or on-premise origin server may deliver the same content far more cheaply.
3-Year TCO: Cloud vs On-Premise (Stable 100-Node Compute Cluster)------------------------------------------------------------------- Cloud (AWS) On-PremiseHardware $0 $1,500,000 (one-time)Compute cost $2,400,000 $0Storage $300,000 $200,000 (hardware)Power/Cooling $0 $180,000Network (egress) $320,000 $50,000 (ISP)Operations staff $0 $240,000 (0.5 FTE) ---------- ----------3-Year Total: ~$3,020,000 ~$2,170,000
Savings on-prem: ~$850,000 over 3 years(Plus: you own the hardware at year 3)The numbers change substantially for smaller, variable workloads where cloud wins clearly. The example above is specifically for large, stable, continuous workloads.
Operational Control and Security Audit Requirements
Certain security and compliance frameworks require a level of physical access control and audit capability that cloud environments cannot satisfy. FedRAMP High (US government systems handling sensitive national security data), some Department of Defense requirements, and intelligence community workloads specify that the organisation must have exclusive physical control of hardware. Shared physical infrastructure — even with strong logical isolation — does not satisfy these requirements.
For these workloads, on-premise or co-located infrastructure in a facility the organisation owns or exclusively occupies is not a preference but a compliance requirement.
The cloud is an excellent answer for most workloads. The word “most” is doing important work in that sentence.