Following our successful collaboration with a leading Thai jewelry manufacturer, where we reduced operational costs by 81% using Azure Spot VMs for their AI-powered video processing system, Gart Solutions has once again partnered with a client to optimize their cloud infrastructure. In the previous case study, Cutting Costs by 81%: Azure Spot VMs Drive Cost Efficiency for Jewelry AI Vision, we demonstrated how utilizing Azure Spot Virtual Machines significantly reduced costs for processing large volumes of data.
In this new case, we faced a different challenge: our client experienced an unexpected spike in costs. Through a deep investigation, we identified that inefficient application resource usage, particularly involving high-cost storage operations, was driving up expenses. This case study highlights how we identified the root cause and optimized their cloud infrastructure for cost efficiency once again.
Background
Our client approached us to set up their cloud infrastructure, which was implemented smoothly. However, within a few weeks, they noticed that their costs were rising significantly more than anticipated. Their cloud expenses soared to $4500, which was far above the initial cost projections we had discussed.
The unexpected surge in costs was not related to virtual machine expenses, which we initially assumed would be the largest component. Instead, a deep dive into their Azure billing revealed that the majority of their expenses—around 85%—were due to storage operations.
Identifying the Problem
To identify the root cause of the cost surge, we turned to the Azure Pricing Calculator to cross-reference the volume of data currently stored. The initial review showed that the storage costs alone, even with the 17TB of data, should not have led to such a significant increase.
Data transfer operations. While each operation appeared inexpensive at just $0.05 per 10,000 transactions, the situation was unusual due to a massive spike in the number of operations. Ordinarily, operation costs should not exceed storage costs unless there is an anomaly, and this case was exactly that.
The Investigation
Through detailed analysis, we uncovered that the client’s storage system was processing an abnormal number of operations. The data showed 1.15 billion operations over a short period. A closer look revealed that these operations were primarily “list blobs” operations, totaling approximately 700 million requests.
By plugging these numbers into the Azure Pricing Calculator, we confirmed that these operations alone accounted for the unexpected $3,800 cost surge.
Root Cause and Solution
The root cause of the problem was the client’s application, which was continuously checking for new files in the storage. This behavior led to an enormous number of “list blobs” requests, all of which carried a cost. The application was inefficiently designed, leading to unnecessary operations and inflated costs.
To address this issue, we provided the following recommendations:
Optimizing the Application: The development team was tasked with reworking the application’s file-checking mechanism to reduce the frequency and volume of these expensive operations.
Monitoring Costs and Usage Patterns: The case revealed that cloud infrastructure costs are not solely dependent on the infrastructure itself. The way an application interacts with cloud resources can drastically influence expenses. Monitoring usage patterns is essential to avoid surprises.
Cost Alerts and Budgeting: Going forward, we advised the client to set up cost alerts and establish monthly budgets. This would ensure that any unusual usage spikes, like the one we encountered, are flagged early before they lead to substantial overages.
