In this video, Sharon discusses high performance computing (HPC) and big compute, outlines popular use cases, and explores three common Microsoft architectures.
- [Instructor] Let's start this course by exploring High Performance Computing otherwise known as HPC Designs. But first, let's quickly define HPC. Microsoft defines HPC as large-scale cloud computing power on demand. You may also hear it referred to as big compute. Typical uses for HPC or big compute include rendering of video graphics, for example think about a Disney movie or an Xbox game, media transcoding, fluid dynamics testing and simulations, for example testing the airflow around an airplane, genomics, and finally image analysis and processing.
All of these cases are extremely compute intensive. Our first design example is a big compute or HPC as a service and don't get worried if some of the services that I talk about here are unfamiliar at this point. We'll be covering these services during the course. Here we start off with a web app or client app which then sends data to Azure Batch which in turn sends out into a pool in Azure and this pool could contain not one or two or six virtual machines, but tens of thousands of virtual machines.
And don't forget, Microsoft loves Linux so these virtual machines could be either a Windows virtual machine or a Linux virtual machine. And finally we have some storage that can be used to store the unstructured data or any of the executable files that are required by our HPC or big compute applications. Next, we have the HPC cluster in the cloud. Here you will notice that we have replaced the web app and client app and Batch with ARM templates and scripts.
The pool is now leveraging the virtual network which contains the virtual machines or virtual machine scale sets. An HPC cluster head node is required in this architecture. You will also need to use A8, A9, and H Series virtual machines because these virtual machines have RDMA networking and RDMA networking allows for high bandwidth and microsecond latencies between the nodes, perfect for our clusters.
These virtual machines are also in an availability set to ensure that they're always available. And finally we have an on-premise HPC environment bursting to Azure. In this example, we're going to leverage the virtual machines in Azure for the compute-intensive node and we connect to Azure either using Express Route or a VPN network. Keep in mind, there are other compute-intensive applications including the Azure worker role, HPC services for Excel, MPI or Message Passing Interface applications, and Java on a virtual machine.
Back in the day, HPC or big compute used to only be available for universities or corporations that had the compute power available to them. Now, big compute is available for everyone in Azure.
- Creating compute-intensive applications
- Creating long-running applications
- Implementing messaging systems
- Azure Service Bus relays
- Using Azure Storage queues
- Creating an Azure Event Hub
- Creating Azure WebJobs
- Managing cloud environments with Azure Active Directory Domain Services