Workload Placement: Putting Your AI Where Your Data Resides
Workload placement decisions are rarely static. As healthcare organizations mature in their use of AI, they often revisit where applications run based on changing cost structures, performance needs and governance requirements.
“Workload placement is going to take into consideration things like latency as well as accessibility,” says Eryn Brodsky, server and storage practice lead at CDW. “Organizations need to think about what AI outcomes they want to leverage the data. You need to have quick access to it, but you also have to ensure that you have proper access to it.”
That balancing act frequently leads to movement between environments over time. Many organizations begin their AI journeys in the cloud, drawn by its flexibility and access to large-scale compute. But as workloads stabilize and costs become clearer, some shift those workloads back into their own environments.
“It’s not uncommon for us to see customers starting in the cloud, as the majority of our customers are, then bringing those applications back on-premises. We call it repatriation,” Brodsky says. “They repatriate those workloads on-prem because they have to consider the cost of maintaining those applications and those workloads in the cloud versus the cost of being able to invest in the architecture to support them.”
Cost optimization is only one piece of the equation. Performance and efficiency also play a major role, particularly as AI workloads scale and demand more specialized infrastructure.
“What we’re looking for is lowest cost per function or lowest cost per token,” Gutierrez says. “Leveraging all of the available optimizations — whether it’s through power and cooling, accelerated infrastructure, better data pipelines, better data posture — is really important to achieving the desired outcomes, reducing latency and getting users the information that they need when they need it.”
Those optimizations increasingly depend on how well organizations align their compute resources to the specific needs of AI workloads. In particular, the relationship between CPUs and GPUs has become a critical factor in modern data center design.
“Advancements in accelerated compute are really important when thinking about the data center of the future,” Gutierrez says. “There are certain applications and certain data center processes that simply work better on a GPU. So, understanding the GPU-CPU relationship in your data center and what workloads or what jobs you can assign to each of those is really important in optimizing your infrastructure.”
How Microsoft Azure Local Helps Healthcare Organizations With AI
As health systems navigate these decisions, platforms that unify management across environments are becoming increasingly valuable. Microsoft’s Azure Local offers one such option, enabling organizations to build and scale AI infrastructure while maintaining flexibility in where workloads run.
“Organizations need an infrastructure that can grow with their needs, and with Azure Local, you can do that,” Carro says. “You can begin small, and then you can add modularly to satisfy the needs of your artificial intelligence.”
A key advantage of Azure Local is its ability to provide consistent management across hybrid environments. With centralized oversight, IT teams can more easily control resources, enforce policies and ensure workloads are running in the most effective locations.
“Centralized control through Azure Arc is going to give us the way to manage our GPUs, our policies and our workloads across on-premises and the cloud,” Carro says. “This makes it easier for IT teams to scale artificial intelligence consistently without having operational problems, because we are watching everything on the same control panel.”
