Undoubtedly, AI is our future—which means it’s past time to integrate machine learning models into your FinOps multi-cloud tech stack. AI turns simple tasks into something that can be executed at the click of a button. With well-trained models, FinOps, MSPs, and Enterprises can automate cost detection, forecasting, and anomaly identification, streamlining complex financial operations without increasing their workforce.
The good news? If you’re an Azure user, you can use their Azure Machine Learning feature to stay ahead.
The bad news? The Azure ML pricing structure, like all Azure pricing, can be a bit… complicated.
But don’t worry! We’re something of Azure experts. Read on to avoid any and all monthly budget surprises while maintaining optimal customer experiences.
What is Azure Machine Learning?
Azure Machine Learning (ML) is an open, interoperable platform that streamlines the process of building, training, and deploying machine learning models, helping you optimize your multi cloud resources and manage costs efficiently in alignment with FinOps best practices. For teams seeking flexibility in discovering new project assets and resources while easily sharing existing files, Azure ML serves as a pivotal tool for collaboration.
Source: Microsoft
Azure Machine Learning (ML) is recognized for its strong security features. It works well with other Azure services to keep all ML workflows secure.
Such as:
- Azure Key Vault securely manages and stores critical information such as API keys and credentials.
- Azure Container Registry ensures the safe management of container images, maintaining isolation and safety for machine learning environments.
- Azure Virtual Networks enable you to segregate machine learning projects within your network, fostering a secure and collaborative space for your ML tasks.
Who uses Azure ML for FinOps purposes?
Azure Machine Learning is the perfect tool for FinOps groups or individuals who want to start integrating machine learning processes into their multi cloud tech stack. This tool allows you and your team to focus on what you do best while Azure ML handles menial, automatable tasks.
Source: Microsoft
Wait, it gets better. ML integrates well with any other tools you use in the Microsoft Azure cloud ecosystem, so optimizing security networks or role-based controls is easy.
In other words, if you want to enhance your FinOps engineer’s daily work, Azure ML will appeal quite nicely!
David Drai
CEO & Co-Founder, Anodot
TIPS FROM THE EXPERT
1. Use burstable VMs for non-critical workloads
Opt for burstable VM instances (e.g., B-series VMs) for model development or lightweight training tasks. These VMs can save significant costs by allowing you to leverage CPU bursting when needed, while keeping the base price low.
2. Preemptive VM usage for training
Utilize low-priority (preemptible) VMs for training ML models, especially for non-urgent tasks. This saves up to 80% on compute costs, though jobs may be interrupted and require rescheduling, making this ideal for fault-tolerant workloads.
3. Leverage spot pricing for experimentation
When running experiments or training smaller models that can tolerate interruptions, spot pricing offers a discounted compute option, allowing you to reduce your ML cost.
4. Auto-scale compute clusters
Set your compute clusters to auto-scale based on demand, with a low minimum node count. This way, clusters spin up only when needed, avoiding unnecessary charges when idle and optimizing resource use.
5. Use incremental storage strategies
Regularly clean up unused datasets, training logs, and outdated models from Azure Blob Storage. Enable tiered storage (e.g., Cool or Archive tiers) for long-term, infrequently accessed data to drastically reduce storage costs.
How to set up your new Azure Machine Learning workspace
Source: Microsoft
Setting up your Azure ML isn’t as hard as you think. Follow these steps, and you should be good to go:
-
- Sign into your Azure Portal account. Or Create an account if you don’t already have one.
- Search for “Machine Learning”. Select it from the other services.
- Hit “Create” to start a new Machine Learning workspace.
- Select the basic settings for Subscription, Resource Group (either pick an old or pre-established one), Workspace Name, and Region (pick either your region or one close to you).
- Pick your resource details for Storage Account (make a new one or use a pre-existing one), Key Vaulted, Container Registry. You can also opt into Application Insights for monitoring resources.
- Review your choices to make sure everything is accurate.
- Deploy your new ML workspace!
How are Azure Machine Learning costs calculated?
Before we get into these numbers, keep in mind that these prices have been calculated assuming the user is based in North America, so it’s possible your costs might be higher or lower than the numbers below. Look to see how prices might vary for other parts of the world here.
Now that that’s been covered, let’s break down how your Azure ML bill works.
There are four main factors that contribute to costs:
- VNets and load balancers. The more cluster support you need, the higher your bill.
- Compute time. Anything from profiling a data set to deployed models or real-time endpoints on Azure Kubernetes services can contribute.
- Storage. Anything from storage for trained models, metrics, or logs will add to your total.
- Azure container registry. Yes, you’ll need to pay for your registered containers.
The key to keeping your Azure ML pricing low is optimizing everything to the fullest and making sure you have the best possible tools to track any changes.
Pro tips to manage Azure Machine Learning costs
Keep your Azure ML costs low while maintaining quality customer experiences by paying close attention to the following factors:
Optimizing Compute costs
As you set up your compute cluster, you must select the best compute resource for your experiments.
It may surprise you to learn that the bulk of your bill won’t come from compute costs and training r models. The actual training process makes up only a small amount of the costs – though this can vary from user to user. If you’re expecting heavy training runs, prepare to invest more.
Here are our four tips to handle compute usage if you intend on using Azure ML for training large models:
- Don’t pick a super low compute tier. If you pick something too low, it will likely save you more money in the short run, but because you’re stuck with slower processing time, it’ll cost more in the long run in resources and time.
- Specify 0 as the minimum number of nodes for your compute cluster. This means your compute resources can shut off when you no longer have any active work scheduled, letting you dodge additional resource charges.
- Use low-priority compute resources during training tasks. If you don’t mind training tasks taking a bit longer or having to be restarted if there’s limited capacity, your experiments are a great place to save money.
- Enable an idle shutdown timer. Set a stop compute instance schedule for off-hours. This means you don’t have to worry about hidden-away compute instances in notebooks leading to surprise charges.
The key here is to maintain a quality offering while eliminating waste. We’ll explain how to do that below.
Monitoring Storage prices
Azure Storage is the most common budget-killer when it comes to ML pricing. Make sure to delete any trained models you no longer use . It’s best to regularly audit any stored data on Azure so you’re not paying for something that isn’t useful.
The following are the biggest contributors to increased storage:
- Log Model metrics
- Data profiles
- Training data
- Trained models
For instance, when automated ML trains your models to identify the most effective hyperparameters, you’ll achieve a highly efficient model. However, this process may also leave you with numerous underperforming models stored away, increasing your storage costs over time.
Managing Endpoints
Endpoints are another pain point for Azure ML pricing. Deploying real-time models to live endpoints is a powerful feature… which means it’s also very expensive. You’ll have to pay for Azure Kubernetes Services resources or Azure Container Instances and associated container registries, storage, and load balances. You’re also on the hook for all sorts of real-time costs 24/7 – always on cost, autoscaling costs, and idle costs, so plan carefully.
Here’s how you can optimize:
- Azure Container Instances (ACI) are usually less expensive than Azure Kubernetes Services (AKS) since AKS clusters are made for product-level tasks, while ACIs are more for developing and testing. So, if you’re using Azure ML for testing and development, it’s a good idea to use ACI to save on costs!
- Use batch endpoints instead of real-time endpoints to lower compute costs when you’re able.
- Remove endpoints you aren’t regularly using to help save costs while maintaining UI.
Azure Container Registry
Azure Container Registry (also known as ACR) is where you store, build, and manage your container images. It enables you to replicate images across multiple locations and provides added security by offering image signing through Docker Content Trust.
You must create various resources in your Azure Machine Learning Workspace, but the container registry is optional. Since it comes with an associated fixed cost, opt out of it for now or use a pre-existing container registry even if you’re not actively using it.
If you ever deploy a container, you won’t need to worry about anything going wrong because Azure ML Workspace will automatically create a container registry. So don’t make one unless you need it!
How to track Azure ML pricing
Microsoft does provide some tools to help you monitor changes in Azure pricing. You can use Azure Cost Management to monitor cost alerts and changes to spend. There’s also their Pricing Calculator that you can use to project how much service add-ons might cost.
However, these tools have limitations. Though you can pull your Azure AI and ML services into the same dashboards to project costs with Azure’s tools, you often won’t get a full view of your multi cloud experience or an in-depth analysis of how to address pricing issues. You won’t get the best view into how your resources might be going to waste, or how to optimize your customer’s user experience best while maintaining profitable margins.
Source: Microsoft
Top solution to track Azure ML spend
What is the best solution for keeping track of your Azure ML spend?
A third-party tool that works alongside you to help reduce Azure ML pricing without any ulterior motives to increase your Azure costs.
And we’re the cloud cost optimization tool. Anodot can help you save up to 40% on annual spending.
Anodot lets you get all multi cloud data in one place. Picture this: a UI-friendly dashboard that shows where all of your spend is going and fluctuations captured down to the hour with retention periods up to 18 to 24 months. Finally, you can finally have that 100% visibility into your cloud performance that you’ve always dreamed of.
Why Anodot? We’ve been working to demystify cloud costs for FinOps organizations for 10 years.
Other Anodot features include:
- Real-time anomaly detection: Automated alerts that improve response time to cloud spend spikes and allow you to track VM, GPU, cluster, and other training and deployment resource-associated costs.
- Customizable alerts: Anodot allows you to set up custom daily, weekly, or monthly alerts based on spending thresholds, which means you will be notified when your Azure ML costs get out of hand.
- AI-powered feedback: Budgeting has never been easier with our CostGPT, which informs your decisions with rapid, AI-powered recommendations. Reveal immediate insights into hidden expenses, pricing inefficiencies, unused resources, and more.
- Comprehensive multi cloud visibility: Full support and visibility across all cloud platforms so you can see your cloud spend and activity all in one place.
- Cost-saving Recommendations: Anodot’s recommendations cover a variety of Azure services, including Disk, VM, MySQL, SQL Data Warehouse, PostgreSQL, Cosmos DB, Maria DB, Load Balancer, Snapshots, Data Explorer, Redis, Kusto, RI Commitments, and App-Service.
Want a proof of concept? Talk to us to learn how much you can save with Anodot’s tools.
Written by Perry Tapiero
You'll believe it when you see it
