High Performance Computing (HPC)

In the past, high performance computing (HPC) required costly supercomputers that typically required whole buildings to house them, huge amounts of power and cooling to run, and whole teams of engineers to write programs for them. Today, new power-efficient, multi-core platform technology offers the possibility of using HPC in any environment, from clustered platforms in the enterprise data center to four or eight-processor systems for researchers, analysts, or designers.

The biggest challenge to creating these new, mainstream HPC systems is developing software that can take advantage of all the power of the new platforms. HPC requires parallel computing that can be complex and costly to develop, debug, deploy, and maintain. The clusters typically used for HPC can be equally complex to architect and operate.

What will bring HPC to the mainstream are development tools that can automate the process of parallelizing applications, allowing developers to provide value through their domain expertise, and clustering solutions that streamline and simplify the process of building and maintaining compute clusters. Together, Dual-Core and Quad-Core Intel® Xeon® processors and Microsoft® Windows® Compute Cluster Server 2003 (CCS) offer fast HPC development plus outstanding performance and reliability.

The Definition of HPC
HPC involves using a collection of computing resources that cooperate to solve a problem involving large amounts of data and calculation. The simplest and most cost-effective platform for HPC applications is to use a cluster of connected, independent multi-core and multi-threaded computers. The computing approach may be “tightly coupled,” as in symmetric multi-processing, where computing elements do synchronized work on parts of the same task, or more “loosely coupled,” like the computers in a cluster that work independently and synchronize only as required by the computing problem (for example, to compile the sum of independent calculations).

HPC generally demands specialized program optimizations to get the most from a system in terms of input/output, computation, and data movement. Some of these optimizations are handled by the commercial, scientific or other HPC applications, and some must be handled by the supercomputer operating system or the cluster server.

The Payoff of HPC
The payoff of HPC depends on the number and performance of available computing resources and the developer’s ability to partition a computing problem or set of tasks to take advantage of the available resources. By parallelizing a problem it can be solved more quickly and efficiently. The “speed-up” gained through parallelization is essentially a function of the number of nodes available to share the work to solve a given problem, their speed, and the percentage of calculations that can be formed in parallel vs. the percentage that must be performed serially. This concept is known as Amdahl’s law.

Parallelization can happen at many levels: OS, clustering, the application, etc. In most organizations, developers have acquired a great deal of expertise in their scientific, engineering, or business domain. Parallel programming and cluster management for HPC requires a different set of expertise. It can be time-consuming and expensive to build, test and optimize parallel applications from the ground up. Organizations will achieve the fastest and highest payoff from HPC by choosing platforms that simplify and automate the process as much as possible.

Getting the Most from Compute Clusters
Clustered platforms are the most cost-effective way to do HPC, especially if the clusters are built from affordable, industry-standard components that allow developers and system managers to leverage existing skills and infrastruc¬ture. There are three ways to get the most from HPC clusters:

• Choosing robust, high performance platforms for the nodes in your cluster
• Minimizing Total Cost of Ownership (TCO)—the cost to deploy, run, manage, and expand the HPC system—by choosing a clustering solution that is simple to set up and manage
• Maximizing utilization through robust, efficient clustering software and parallel programming

For example, the combination of Intel Xeon processors and CCS meets these goals with outstanding HPC performance and industry-leading reliability—clustering that is simple to deploy, operate, and integrate with existing infrastructure, and integrated parallel programming tools to accelerate the programming and deployment process.

Choosing Your Nodes
Processors are the foundation of your HPC system. When you choose a clustering platform, look for products that offer:

• Balanced performance. Performance affects both speed-up and solution costs. In choosing a cluster platform for your HPC application, consider raw performance for your application, and also look at price/performance and performance/watt and how they will affect TCO. Choose the most affordable platform that will meet your current requirements and scale to meet your future HPC needs
• Best-in-class reliability. System downtime adds to your time-to-solution, and computer worms, viruses or data errors can also affect the results of complex applications that may take hours or even days to re-run. To prevent these problems, choose cluster nodes that offer advanced reliability features such as enhanced memory ECC, sparing and mirroring for data protection and availability, and symmetric CPU access for fast system restart and/or processor failover
• Widespread vendor relationships that give you choice and flexibility in system components and software
• HPC expertise and comprehensive support for developers creating HPC applications and systems

Transforming Research and Business Computing
High performance computing applications can offer invaluable insights into research, design, and business questions. While HPC has traditionally been the province of large research organizations and corporations, the availability of mainstream HPC solutions opens a vast range of new possibilities:

• Even mid-sized businesses can afford real-time financial analysis and business intelligence applications that can project results of different product mixes and strategies
• Businesses can do richer data mining to identify and respond to customer preferences and market trends, potential security threats, etc.
• HPC creates new opportunities for services using expert systems in industries such as healthcare, entertainment, retailing and travel
• Scientific and engineering organizations, colleges and universities can empower designers and engineers with more power to do simulation and analysis in aerospace, biosciences and pharmaceuticals, and other fields.

Together, CCS and Dual-Core Intel Xeon processor-based server systems offer simple and more affordable HPC solutions to help make mainstream HPC applications possible.